Papers with neural networks
Interpretability and Analysis in Neural NLP (2020.acl-tutorials)
Copied to clipboard
| Challenge: | a tutorial aims to introduce the nascent field of interpretability and analysis of neural networks in NLP . |
| Approach: | This tutorial will introduce the nascent field of interpretability and analysis of neural networks in NLP. |
| Outcome: | This tutorial will introduce the nascent field of interpretability and analysis of neural networks in NLP. |
Computationally Efficient Wasserstein Loss for Structured Labels (2021.eacl-srw)
Copied to clipboard
| Challenge: | Existing approaches to estimate the probability distribution of labels are based on tree-Wasserstein distance. |
| Approach: | They propose a tree-Wasserstein distance regularized LDL algorithm for hierarchical text classification tasks. |
| Outcome: | The proposed method performs well on synthetic and real-world datasets and compares favorably with the Sinkhorn algorithm in terms of computation time and memory usage. |
Towards Opinion Summarization of Customer Reviews (P18-3)
Copied to clipboard
| Challenge: | Existing methods to summarize text are limited to small, homogeneous datasets . authors outline future directions to solve these problems . |
| Approach: | They propose to use neural networks to generate summaries of user-generated travel reviews . they aim to take into account shifting opinions over time and address these issues . |
| Outcome: | The proposed method will make it easier for users of review sites to make more informed decisions. |
Pay-Per-Request Deployment of Neural Network Models Using Serverless Architectures (N18-5)
Copied to clipboard
| Challenge: | Using Amazon’s Lambda service for feedforward evaluation and DynamoDB for word embeddings, we demonstrate a serverless deployment of neural networks for NLP applications. |
| Approach: | They propose a pay-per-request pricing model for neural network deployment in NLP applications using Amazon’s Lambda service for feedforward evaluation and DynamoDB for storing word embeddings. |
| Outcome: | The proposed architecture is scalable and inexpensive. |
AutoNLU: An On-demand Cloud-based Natural Language Understanding System for Enterprises (2020.aacl-demo)
Copied to clipboard
| Challenge: | AutoNLU is an on-demand cloud-based system that enables users to create and edit datasets and train and test different state-of-the-art NLU models. |
| Approach: | They introduce an on-demand cloud-based system that provides an easy-to-use interface . they build powerful keyphrase extraction models that achieve state-of-the-art results . |
| Outcome: | The proposed model achieves state-of-the-art on two public benchmarks and is easy to use and use. |
Variational Inference and Deep Generative Models (P18-5)
Copied to clipboard
| Challenge: | Unsupervised and semi-supervised learning has been addressed scarcely in NLP . this tutorial provides an introduction to variational inference followed by an example-driven discussion of how to use variational methods for training DGMs. |
| Approach: | This tutorial provides an introduction to variational inference followed by an example-driven discussion of how to use variational methods for training DGMs. |
| Outcome: | This tutorial provides an introduction to variational inference followed by an example-driven discussion of how to use variational methods for training DGMs. |
Neural Self-Training through Spaced Repetition (N19-1)
Copied to clipboard
| Challenge: | Existing methods for self-training rely on predetermined policies to sample unlabeled data. |
| Approach: | They propose a semi-supervised learning approach that uses spaced repetition to dynamically sample informative and diverse unlabeled instances with respect to individual learner and instance characteristics. |
| Outcome: | The proposed model outperforms existing semi-supervised learning approaches on publicly-available datasets. |
Transformer-specific Interpretability (2024.eacl-tutorials)
Copied to clipboard
| Challenge: | Transformers are dominant play-ers in various scientific fields, but their inner workings remain opaque. |
| Approach: | This tutorial presents a trending approach to interpreting Transformers . it uses specific features of the Transformer architecture to quantify context- mixing interactions . |
| Outcome: | This tutorial aims to show how a new trending approach can be applied to Transformer-based models. |
Representations of Meaning in Neural Networks for NLP: a Thesis Proposal (2021.naacl-srw)
Copied to clipboard
| Challenge: | Neural networks are the state-of-the-art method of machine learning for many problems in NLP. |
| Approach: | They propose to examine the distribution of meaning in the vector space representation of words in neural networks trained for NLP tasks. |
| Outcome: | The proposed method would be compatible with distributional hypothesis, structuralism, and semantic holism. |
Learning to Imagine: Integrating Counterfactual Thinking in Neural Discrete Reasoning (2022.acl-long)
Copied to clipboard
| Challenge: | Existing NDR models suffer from large performance drop on hypothetical questions, e.g., “what the annualized rate of return would be if the revenue in 2020 was doubled”. |
| Approach: | They propose a learning to imagine module which can be seamlessly incorporated into NDR models to perform the imagination of unseen counterfactual. |
| Outcome: | The proposed model can perform the imagination of unseen counterfactuals on hypothetical questions. |
Automatic Taxonomy Induction and Expansion (D19-3)
Copied to clipboard
Nicolas Rodolfo Fauceglia, Alfio Gliozzo, Sarthak Dash, Md. Faisal Mahbub Chowdhury, Nandana Mihindukulasooriya
| Challenge: | Knowledge Graph Induction Service (KGIS) enables automatic taxonomy induction and human-in-the-loop curation. |
| Approach: | They describe the features of the Knowledge Graph Induction Service (KGIS) KGIS allows the user to semi-automatically curate and expand the induced taxonomies through a component called Smart SpreadSheet . |
| Outcome: | The Knowledge Graph Induction Service (KGIS) is an end-to-end knowledge graph induction system. |
Deep Bayesian Natural Language Processing (P19-4)
Copied to clipboard
| Challenge: | Introduction to deep Bayesian learning for natural language addresses the fundamentals of statistical models and neural networks. |
| Approach: | This tutorial addresses the advances in deep Bayesian learning for natural language . it focuses on advanced Bayessian models and deep models . authors present case studies and domain applications to tackle different issues . |
| Outcome: | This tutorial focuses on advanced Bayesian models and deep models for natural language . case studies and domain applications are presented to tackle different issues in deep Bayessian processing, learning and understanding. |
Start Simple: Progressive Difficulty Multitask Learning (2024.naacl-srw)
Copied to clipboard
| Challenge: | a novel neural network training strategy that trains neural networks using subtasks of progressive difficulty is proposed . this strategy could help us understand how neural networks learn, authors say . |
| Approach: | They propose a multitask learning strategy that employs progressive difficulty subtasks to train neural networks. |
| Outcome: | The proposed strategy can improve model performance across a range of NLP tasks and data sets. |
Modeling Content and Context with Deep Relational Learning (2021.tacl-1)
Copied to clipboard
| Challenge: | Existing frameworks for combining neural and symbolic representations are limited to simple relational learning tasks. |
| Approach: | They propose a declarative framework for specifying deep relational models that integrates expressive language encoders and provides an interface to study the interactions between representation, inference and learning. |
| Outcome: | The proposed framework integrates with expressive language encoders and provides an interface to study the interactions between representation, inference and learning. |
Guiding Variational Response Generator to Exploit Persona (2020.acl-main)
Copied to clipboard
Bowen Wu, MengYuan Li, Zongsheng Wang, Yifu Chen, Derek F. Wong, Qihang Feng, Junhong Huang, Baoxun Wang
| Challenge: | Neural Response Generators (NRGs) use persona information of users to perform personalized conversations . current studies focus on incorporating explicit meta-data of user profiles or character descriptions to generate persona-aware responses. |
| Approach: | They propose to use persona information of users in Neural Response Generators to perform personalized conversations. |
| Outcome: | The proposed method improves persona-aware response generation and the metrics are reasonable to evaluate them. |
Slot-consistent NLG for Task-oriented Dialogue Systems with Iterative Rectification Network (2020.acl-main)
Copied to clipboard
| Challenge: | Existing approaches to natural language generation are prone to errors, such as neglecting input slot values and generating redundant slot values. |
| Approach: | They propose an iterative rectification network to improve general NLG systems . they apply bootstrapping algorithms to sample training candidates and incorporate reward . |
| Outcome: | The proposed methods significantly reduce the slot error rate for strong baselines. |
SyGNS: A Systematic Generalization Testbed Based on Natural Language Semantics (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing models have limitations to generalize to diverse semantic phenomena, and it is unclear whether they can capture compositional meanings. |
| Approach: | They propose a systematic generalization testbed based on Natural language semantics to map natural language sentences to multiple meaning representations. |
| Outcome: | The proposed model can generalize to unseen combinations of quantifiers, negations, and modifiers, but not to the others. |
The DipInfoUniTo Realizer at SRST’19: Learning to Rank and Deep Morphology Prediction for Multilingual Surface Realization (D19-63)
Copied to clipboard
| Challenge: | SR is one of the main tasks involved in Natural Language Generation. |
| Approach: | They propose a system which divides the SR task into two independent subtasks, namely word order prediction and morphology inflection prediction. |
| Outcome: | The proposed system is a direct successor to the architecture presented at SR'19. |
YATO: Yet Another deep learning based Text analysis Open toolkit (2023.emnlp-demo)
Copied to clipboard
| Challenge: | YATO is an open-source toolkit for text analysis with deep learning . it supports free combinations of three types of widely used features . |
| Approach: | They introduce YATO, an open-source toolkit for text analysis with deep learning. |
| Outcome: | YATO is an open-source toolkit for text analysis with deep learning . the toolkit supports free combinations of three types of widely used features . |
Putting the Con in Context: Identifying Deceptive Actors in the Game of Mafia (2022.naacl-main)
Copied to clipboard
| Challenge: | Using neural networks, we can model the impact of speaker role on language use through the game of Mafia. |
| Approach: | They analyze the effect of speaker role on language use through the game of Mafia, in which players are assigned either an honest or a deceptive role. |
| Outcome: | The proposed model outperforms a standard BERT-based text classification approach on two auxiliary tasks and identifies features that distinguish between player roles. |
FiMMIA: scaling semantic perturbation-based membership inference across modalities (2026.eacl-demo)
Copied to clipboard
| Challenge: | Membership Inference attacks aim to determine whether a specific data point was included in the training set of a target model. |
| Approach: | They propose to train a neural network to analyze the target model’s behavior on perturbed inputs, capturing interactions between semantic domains and loss values on members and non-members in the local neighborhood of each sample. |
| Outcome: | The proposed methods can detect distribution shifts in existing datasets and release a baseline pipeline to detect them. |
Dataset Distillation with Attention Labels for Fine-tuning BERT (2023.acl-short)
Copied to clipboard
| Challenge: | Specifically, we propose to introduce attention labels, which can efficiently distill the knowledge from the original dataset and transfer it to the transformer models via attention probabilities. |
| Approach: | They propose to introduce attention labels which can efficiently distill the knowledge from the original dataset and transfer it to the transformer models via attention probabilities. |
| Outcome: | The proposed methods perform impressively in four different NLP tasks and achieve 93.2% accuracy in AGNews, which is 98.5% of the original dataset even with only one sample per class and only one gradient step. |
Adaptive Rank Selections for Low-Rank Approximation of Language Models (2024.naacl-long)
Copied to clipboard
| Challenge: | Singular Value Decomposition (SVD) or its weighted variants has progressed in compressing language models. |
| Approach: | They propose a binary masking mechanism for optimizing the number of ranks in a differentiable framework. |
| Outcome: | The proposed algorithm achieves much better accuracy than previous SVD and its weighted variants. |
CNNs for NLP in the Browser: Client-Side Deployment and Visualization Opportunities (N18-5)
Copied to clipboard
| Challenge: | a JavaScript implementation of a convolutional neural network performs feedforward inference completely in the browser. |
| Approach: | They propose a JavaScript implementation that performs feedforward inference completely in the browser. |
| Outcome: | The proposed model performs feedforward inference completely in the browser without server requests . the proposed model is useful for applications with stringent latency requirements or low connectivity . |
Hacking Neural Evaluation Metrics with Single Hub Text (2026.eacl-short)
Copied to clipboard
| Challenge: | Recent embedding-based neural text evaluation metrics are not reliable due to black-box nature of neural networks. |
| Approach: | They propose to find a single adversarial text in the discrete space that is consistently evaluated as high-quality regardless of the test cases. |
| Outcome: | The proposed method outperforms translations generated individually for each source sentence in English-to-Japanese and English- to-German translation tasks. |
Extract, Transform and Filling: A Pipeline Model for Question Paraphrasing based on Template (D19-55)
Copied to clipboard
| Challenge: | Recent approaches for paraphrasing generate unpredictable results . |
| Approach: | They propose a question paraphrasing pipeline model based on templates that identifies template and retrieves candidate templates and fills them with original topic words. |
| Outcome: | The proposed model outperforms the seq2seq model on two datasets and is more promising when the training sample is small. |
Assessing the Limits of the Distributional Hypothesis in Semantic Spaces: Trait-based Relational Knowledge and the Impact of Co-occurrences (2022.starsem-1)
Copied to clipboard
| Challenge: | a rise in performance in NLP has led to a decrease in interpretability . a recent study examined how neural semantic models capture relational knowledge . |
| Approach: | They evaluate how well English and Spanish semantic spaces capture a particular type of relational knowledge . they also explore the role of co-occurrences in this context . |
| Outcome: | The proposed model can be used to predict traits associated with concepts in English and Spanish. |
EmoNoBa: A Dataset for Analyzing Fine-Grained Emotions on Noisy Bangla Texts (2022.aacl-short)
Copied to clipboard
| Challenge: | EmoNoBa is a dataset for fine-grained emotion detection on Bangla text . it is based on 22698 comments from social media sites on 12 domains . |
| Approach: | They propose a manually annotated dataset of 22,698 Bangla comments from social media sites on 12 different domains to use for fine-grained emotion detection. |
| Outcome: | The proposed dataset of 22,698 public comments on 12 domains shows that hand-crafted features perform better than neural networks and pre-trained language models. |
KERMIT: Complementing Transformer Architectures with Encoders of Explicit Syntactic Interpretations (2020.emnlp-main)
Copied to clipboard
Fabio Massimo Zanzotto, Andrea Santilli, Leonardo Ranaldi, Dario Onorati, Pierfrancesco Tommasino, Francesca Fallucchi
| Challenge: | Syntactic parsers are losing their centrality in downstream tasks due to the success of large-scale textual representation learners. |
| Approach: | They propose to embed symbolic syntactic parse trees into artificial neural networks to visualize how syntax is used in inference. |
| Outcome: | The proposed encoder can visualize how syntax is used in inference. |
Intriguing Effect of the Correlation Prior on ICD-9 Code Assignment (2023.acl-srw)
Copied to clipboard
| Challenge: | The Ninth Revision of the International Classification of Diseases (ICD-9) is a standardized coding system used worldwide to classify and code diseases, injuries, and other health conditions. |
| Approach: | They evaluate the usefulness of correlation bias and suggest it could improve ICD-9 code assignment in some cases. |
| Outcome: | The proposed model improves on classes that are more imbalanced and less correlated with other codes, but the effect on individual class can be negative or positive. |
Neural Segmental Hypergraphs for Overlapping Mention Recognition (D18-1)
Copied to clipboard
| Challenge: | Existing models that assume that mentions are non-overlapping spans in text may not perform well in practice. |
| Approach: | They propose a segmental hypergraph representation to model overlapping entity mentions that are prevalent in many practical datasets. |
| Outcome: | The proposed representation achieves state-of-the-art performance in three benchmark datasets annotated with overlapping mentions. |
Neural Gibbs Sampling for Joint Event Argument Extraction (2020.aacl-main)
Copied to clipboard
| Challenge: | Existing methods for event argument extraction cannot adequately model the correlation between event arguments and their roles. |
| Approach: | They propose a Bayesian model to jointly extract event arguments using Gibbs sampling . they train two neural networks to model prior distribution and conditional distribution over event arguments . |
| Outcome: | The proposed model can achieve comparable results to existing methods on two widely-used datasets. |
Uncertainty Modeling for Machine Comprehension Systems using Efficient Bayesian Neural Networks (2020.coling-industry)
Copied to clipboard
| Challenge: | Neural approaches have improved machine comprehension tasks, but models often operate as a black-box, resulting in lower interpretability. |
| Approach: | They propose a hybrid approach to quantify model uncertainty using Bayesian weight approximation and boost up inference speed by 80% relative to test time. |
| Outcome: | The proposed approach boosts inference speed by 80% relative to the previous approach and is applied to a clinical dialogue comprehension task. |
CytonMT: an Efficient Neural Machine Translation Open-source Toolkit Implemented in C++ (D18-2)
Copied to clipboard
| Challenge: | Neural machine translation (NMT) has made remarkable progress over the past few years. |
| Approach: | They propose to use C++ and NVIDIA’s GPU-accelerated libraries to build an open-source neural machine translation toolkit called CytonMT. |
| Outcome: | The proposed toolkit accelerates the training speed by 64.5% to 110.8% on neural networks of various sizes, and achieves competitive translation quality. |
Generating Inflectional Errors for Grammatical Error Correction in Hindi (2020.aacl-srw)
Copied to clipboard
| Challenge: | Automated grammatical error correction is a data-heavy task . indic languages have a relatively low amount of digitized content and complex morphology . |
| Approach: | They generate a corpus of inflectional errors for training neural networks to correct grammatical errors in Hindi. |
| Outcome: | The proposed model trains on a corpus of inflectional errors extracted from Wikipedia edits. |
Auto-Sizing the Transformer Network: Improving Speed, Efficiency, and Performance for Low-Resource Machine Translation (D19-56)
Copied to clipboard
| Challenge: | Neural sequence-to-sequence models are sensitive to architecture and hyperparameter settings. |
| Approach: | They incorporate architecture search into a single training run through auto-sizing . they show that auto-size can improve BLEU scores by up to 3.9 points . |
| Outcome: | The proposed algorithm improves BLEU scores on low-resource language pairs while removing one-third of the parameters from the model. |
RoFT: A Tool for Evaluating Human Detection of Machine-Generated Text (2020.emnlp-demos)
Copied to clipboard
| Challenge: | Existing studies on how humans perceive machine-generated text are limited due to the prohibitive cost of running human evaluation studies. |
| Approach: | They propose a task to detect the boundary at which a text passage starts off human-written transitions to being machine-generated. |
| Outcome: | The proposed system evaluates machine-generated news articles on a wide range of domains. |
Retrieval, Re-ranking and Multi-task Learning for Knowledge-Base Question Answering (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing work on question answering over knowledge bases limited the search space to a subset of KBs . a retrieval-and-rerank framework is used to access KB and rerank retrieved candidates with more powerful neural networks. |
| Approach: | They propose to share a BERT encoder across all three sub-tasks and define task-specific layers on top of the shared layer. |
| Outcome: | The proposed method improves accuracy and accuracy on the SimpleQuestions dataset and the FreebaseQA dataset. |
Determining Event Durations: Models and Error Analysis (N18-2)
Copied to clipboard
| Challenge: | a crucial piece of information regarding events is their duration, a rarely mentioned attribute . core tasks such as temporal understanding and reasoning would benefit from knowing the expected duration of events. |
| Approach: | They introduce aspectual features that capture deeper linguistic information . they also experiment with neural networks to predict event durations . |
| Outcome: | The proposed models capture deeper linguistic information than previous work and provide useful clues. |
Neural Networks in a Product of Hyperbolic Spaces (2022.naacl-srw)
Copied to clipboard
| Challenge: | Recent advances in the use of hyperbolic spaces have been reported in natural language processing and graph embedding. |
| Approach: | They propose to extend hyperbolic neural networks to a product of hyperbolical spaces by using a single hyperbolically spaced hyperbole. |
| Outcome: | The proposed method improves graph node classification accuracy on tree-like datasets. |
Augmenting Neural Networks with First-order Logic (P19-1)
Copied to clipboard
| Challenge: | Existing paradigms for training neural networks require large datasets, a paper argues . we present a framework for introducing declarative knowledge to neural networks . |
| Approach: | They propose a framework for introducing declarative knowledge to neural networks . they compile logical statements into graphs that augment a network without extra learnable parameters or manual redesign. |
| Outcome: | The proposed framework improves on three tasks, especially in low-data regimes. |
Multiple Character Embeddings for Chinese Word Segmentation (P19-2)
Copied to clipboard
| Challenge: | Chinese word segmentation is regarded as character-based sequence labeling task in most current work but it neglects important fact: Chinese characters contain both semantic and phonetic meanings. |
| Approach: | They propose a shared bi-LSTM-CRF model which fuses linguistic features efficiently by sharing the LSTM network during the training procedure. |
| Outcome: | The proposed model achieves state-of-the-art in AS and CityU corpora without external lexical resources. |
Trace-of-Thought Prompting: Investigating Prompt-Based Knowledge Distillation Through Question Decomposition (2024.acl-srw)
Copied to clipboard
| Challenge: | Trace-of-Thought Prompting allows small neural networks to emulate larger, teacher models with reduced computational demands. |
| Approach: | They propose a framework to distill critical reasoning capabilities from teacher models to student models . they use problem decomposition to enhance interpretability and facilitate human-in-the-loop interventions . |
| Outcome: | a new framework enables small neural networks to emulate the performance of larger, teacher models . it leverages problem decomposition to enhance interpretability and facilitate human-in-the-loop interventions . the proposed framework is available on github.com/trace-of-thought/trac-of_thought-prompting/main . |
Conversation Learner - A Machine Teaching Tool for Building Dialog Managers for Task-Oriented Dialog Systems (2020.acl-demos)
Copied to clipboard
Swadheen Shukla, Lars Liden, Shahin Shayandeh, Eslam Kamal, Jinchao Li, Matt Mazzola, Thomas Park, Baolin Peng, Jianfeng Gao
| Challenge: | a wide variety of tasks have created a need for flexible task-oriented dialog systems . dialog flows are intuitively interpretable but lack the flexibility needed to handle complex dialogs . |
| Approach: | They propose a machine teaching tool for building dialog managers using familiar tools . they convert the dialog flow into a parametric model and use user-system dialog logs as training data . |
| Outcome: | The proposed tool combines the best of both approaches to build dialog managers . it converts the dialog flow into a parametric model and improves it over time . |
Simplified Neural Unsupervised Domain Adaptation (N19-1)
Copied to clipboard
| Challenge: | Existing unsupervised domain adaptation methods use neural networks to learn representations that are trained to predict the values of subset of important features called “pivot features.” |
| Approach: | They propose to combine the representation learner and task learner to improve on existing neural domain adaptation algorithms by removing heuristically-selected "pivot features" they show competitive performance with a simpler model. |
| Outcome: | The proposed model outperforms existing models by removing heuristically-selected pivot features. |
Quantifying the Hyperparameter Sensitivity of Neural Networks for Character-level Sequence-to-Sequence Tasks (2024.eacl-long)
Copied to clipboard
| Challenge: | Neural networks are notoriously hard to interpret and slightly mysterious to researchers and practitioners alike. |
| Approach: | They formalize hyperparameter sensitivity using two metrics: similarity-based sensitivity and performance-based-sensitivity. |
| Outcome: | The transformer is more sensitive to hyperparameters according to both metrics, but not batch size . large models, multilinguality of NLP models and tasks make hyperparametric tuning more expensive . |
Improving Biomedical Abstractive Summarisation with Knowledge Aggregation from Citation Papers (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing language models struggle to generate technical summaries that are on par with those produced by biomedical experts due to the lack of domain-specific background knowledge. |
| Approach: | They propose a attention-based citation aggregation model that integrates domain-specific knowledge from citation papers and a large-scale biomedical summarisation dataset to build on. |
| Outcome: | The proposed model outperforms state-of-the-art approaches and achieves substantial improvements in biomedical abstractive summarisation. |
Efficient Diagnosis Assignment Using Unstructured Clinical Notes (2023.acl-short)
Copied to clipboard
| Challenge: | Electronic phenotyping entails using electronic health records (EHRs) to identify patients with specific clinical outcomes and determine when those outcomes occurred. |
| Approach: | They propose a framework for electronic phenotyping that integrates labeling functions and a disease-agnostic neural network to assign diagnoses to patients. |
| Outcome: | The proposed framework disambiguates hypertension true positives and false positives with a supervised area under the precision-recall curve (AUPRC) of 0.85. |
Investigating Capsule Network and Semantic Feature on Hyperplanes for Text Classification (D19-1)
Copied to clipboard
| Challenge: | Various neural networks are designed for text classification on the basis of word embedding, but polysemy is a fundamental feature of the natural language, which brings challenges to text classification. |
| Approach: | They propose to use capsule networks to construct the vectorized representation of semantics and utilize hyperplanes to decompose each capsule to acquire the specific senses. |
| Outcome: | The proposed model extracts more discriminative semantic features and yields significant performance gain compared to baseline methods. |
Recurrent Entity Networks with Delayed Memory Update for Targeted Aspect-Based Sentiment Analysis (N18-2)
Copied to clipboard
| Challenge: | Recent work on target-dependent biLSTMs has shown that they are ineffective in aspect-based sentiment analysis. |
| Approach: | They propose a novel architecture that uses external memory chains with a delayed memory update mechanism to track entities. |
| Outcome: | The proposed model improves on a TABSA task using external memory chains with a delayed memory update mechanism. |
Hierarchical Multi-label Classification of Text with Capsule Networks (P19-2)
Copied to clipboard
| Challenge: | In hierarchical multi-label classification, samples are classified into one or multiple class labels organized in a structured label hierarchy. |
| Approach: | They apply and compare shallow capsule networks for hierarchical multi-label text classification and introduce a new real-world scenario dataset. |
| Outcome: | The proposed model outperforms neural networks and non-neural network architectures on a real-world scenario dataset. |
Minimum Description Length Recurrent Neural Networks (2022.tacl-1)
Copied to clipboard
| Challenge: | We show that neural networks that optimize a minimum description length score master memory challenges and perform addition with 100% accuracy. |
| Approach: | They train neural networks to optimize a Minimum Description Length score . they show that they master tasks involving memory challenges and perform addition . |
| Outcome: | The proposed models master languages and perform addition with 100% accuracy . they show that they can generalize from small training corpora and large training corpus . |
Mining Possessions: Existence, Type and Temporal Anchors (N18-1)
Copied to clipboard
| Challenge: | Existing annotations for possession relations can be used to predict possession existence, possession type and temporal anchors. |
| Approach: | They propose to use text annotations to mine possession relations from text . they assign temporal anchors indicating when possession holds between possessor and possessee . |
| Outcome: | The proposed task can predict possession existence, possession type and temporal anchors, and it can be automated. |
Strong Baselines for Simple Question Answering over Knowledge Graphs with and without Neural Networks (N18-2)
Copied to clipboard
| Challenge: | Existing work on simple question answering over knowledge graphs involves increasingly complex NN architectures. |
| Approach: | They propose to decompose the problem into entity detection, entity linking, relation prediction, evidence combination and heuristics. |
| Outcome: | The proposed approach outperforms existing models and benchmarks on a simple QA task. |
On the Mathematical Relationship Between Layer Normalization and Dynamic Activation Functions (2026.eacl-short)
Copied to clipboard
| Challenge: | Layer normalization (LN) is an essential component of modern neural networks. |
| Approach: | They propose a dynamic activation function called Dynamic Tanh which is based on the LN variant RMSNorm and decouples in derivative space. |
| Outcome: | The proposed function reproduces the normalization effect on outliers more accurately than RMSNorm by using a well-defined decoupling procedure in derivative space and an approximation. |
Self-regulation: Employing a Generative Adversarial Network to Improve Event Detection (P18-1)
Copied to clipboard
| Challenge: | Recent studies show that neural networks can be used for event detection but can be contaminated by spurious features. |
| Approach: | They propose a self-regulated learning approach by utilizing a generative adversarial network to generate spurious features. |
| Outcome: | The proposed method is highly effective and adaptable on the ACE 2005 and TAC-KBP 2015 corpora. |
OoMMix: Out-of-manifold Regularization in Contextual Embedding Space for Text Classification (2021.acl-long)
Copied to clipboard
| Challenge: | Recent studies on neural networks with pre-trained weights focus on low-dimensional subspace where the embedding vectors computed from input words are located. |
| Approach: | They propose an approach to find and regularize the remainder of the space, referred to as out-of-manifold, which cannot be accessed through the words. |
| Outcome: | The proposed approach is able to fine-tune the out-of-manifold embedding space on text classification benchmarks. |
The Devil is in the Detail: Simple Tricks Improve Systematic Generalization of Transformers (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies show that basic configurations can improve the performance of neural networks on systematic generalization. |
| Approach: | They propose to revisit basic configurations to improve the performance of Transformers on systematic generalization by revisiting scaling of embeddings, early stopping, relative positional embeddment, and Universal Transformer variants. |
| Outcome: | The proposed models improve accuracy from 50% to 85% on the PCFG productivity split and from 35% to 81% on COGS. |
How Fast can BERT Learn Simple Natural Language Inference? (2021.eacl-main)
Copied to clipboard
| Challenge: | Efficiency of learning of BERT is very slow due to hidden dataset bias . however, some studies show that it can learn with surface clues/patterns . |
| Approach: | They propose to use a simple entailment judgment case to test whether BERT can learn without hidden dataset bias. |
| Outcome: | The proposed case shows that BERT can learn without hidden bias without utilizing dataset bias. |
Detecting Compositionally Out-of-Distribution Examples in Semantic Parsing (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Neural network models suffer from performance losses when faced with compositionally out-of-distribution data. |
| Approach: | They propose to use neural semantic parsers to detect compositionally out-of-distribution (OOD) data. |
| Outcome: | The proposed methods perform well on the standard SCAN and CFQ datasets. |
Deep Neural Models for Medical Concept Normalization in User-Generated Texts (P19-2)
Copied to clipboard
| Challenge: | a medical concept normalization problem is a challenge since social media texts are ambiguous and noisy . a recent study shows that neural architectures leverage the semantic meaning of the entity mention . |
| Approach: | They propose to map a health-related entity mention to a controlled vocabulary . they use powerful neural networks and contextualized word representation models . |
| Outcome: | The proposed model outperforms existing state-of-the-art models in mapping medical concepts to medical terms . the proposed model is based on recurrent neural networks and contextualized word representation models . |
Truth-Conditional Captions for Time Series Data (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing models with attention mechanisms can generate fluent descriptions of salient patterns in time series, but they often generate factually incorrect descriptions. |
| Approach: | They propose a model which first runs small learned programs on the input time series, then identifies the programs/patterns which hold true for the given input, and finally conditions on *only* the chosen valid program to generate the output text description. |
| Outcome: | The proposed model extracts high-level patterns from the data and generates high precision captions even though it is built on a small space of modules. |
A Joint Learning Approach based on Self-Distillation for Keyphrase Extraction from Scientific Documents (2020.coling-main)
Copied to clipboard
| Challenge: | Existing methods for keyphrase extraction are limited by the number of annotated documents. |
| Approach: | They propose a joint learning approach that uses the idea of self-distillation to extract keyphrases from unlabeled articles. |
| Outcome: | The proposed approach outperforms baseline models on two public benchmarks: Inspec and SemEval-2017. |
Exploration and Exploitation: Two Ways to Improve Chinese Spelling Correction Models (2021.acl-short)
Copied to clipboard
| Challenge: | Experimental results show that a sequence-to-sequence learning framework with neural networks can be effective for Chinese Spelling Correction (CSC) |
| Approach: | They propose a sequence-to-sequence learning framework with neural networks that generates more valuable training instances and adds task-specific examples to enhance the model. |
| Outcome: | The proposed method improves generalization and robustness of multiple CSC models across three datasets. |
NeuReduce: Reducing Mixed Boolean-Arithmetic Expressions by Recurrent Neural Network (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Mixed Boolean-Arithmetic (MBA) expressions are difficult to simplify because of interleaving bitwise and arithmical operations. |
| Approach: | They propose a method to learn and reduce MBA expressions using a string to string method . they propose to use a dataset to train the method to reduce MBA rules . |
| Outcome: | The proposed method outperforms all other tools in terms of accuracy, solving time, and performance overhead. |
Extracting Possessions from Social Media: Images Complement Language (D19-1)
Copied to clipboard
| Challenge: | Existing studies show that authors of tweets possess objects they tweet about. |
| Approach: | They propose a dataset and experiments to determine whether tweet authors possess objects they tweet about. |
| Outcome: | The proposed strategy incorporates visual information into any neural network beyond weights from pretrained networks. |
Fast Abstractive Summarization with Reinforce-Selected Sentence Rewriting (P18-1)
Copied to clipboard
| Challenge: | Empirically, we achieve the new state-of-the-art on all metrics (including human evaluation) on the CNN/Daily Mail dataset, as well as significantly higher abstractiveness scores. |
| Approach: | They propose a sentence-level policy gradient method that bridges computation between two neural networks in a hierarchical way while maintaining language fluency. |
| Outcome: | The proposed model achieves state-of-the-art on all metrics and higher abstractiveness scores on the CNN/Daily Mail dataset and faster training convergence than previous models. |
On the Impact of Temporal Representations on Metaphor Detection (2022.lrec-1)
Copied to clipboard
| Challenge: | State-of-the-art approaches for metaphor detection compare their literal - or core - meaning and their contextual meaning using neural networks. |
| Approach: | They propose to use temporal and static word embeddings to account for different representations of literal meanings to examine metaphor detection tasks. |
| Outcome: | The proposed method outperforms static methods but may provide representations of the core meaning of the metaphor too close to their contextual meaning, causing confusion. |
Unit Testing for Concepts in Neural Networks (2022.tacl-1)
Copied to clipboard
| Challenge: | Existing theories of language and cognition hold that these representations are structured in a compositional way and that the meanings of composite concepts (''gray car'') are inherited predictably from the meaning of the parts. |
| Approach: | They propose to test models for determining whether a system’s behavior is consistent with several key aspects of Fodor’s criteria. |
| Outcome: | The proposed models succeed on tests of groundedness, modularity, and reusability of concepts, but important questions about causality remain open. |
Structural generalization in COGS: Supertagging is (almost) all you need (2023.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies have shown that neural networks fail to generalize on out-of-distribution examples. |
| Approach: | They extend a neural graph-based parsing framework to address compositional generalization limitations . they introduce a supertagging step with valency constraints and reduce the graph prediction problem . |
| Outcome: | The proposed approach improves results on COGS datasets that require structural generalization. |
Learning Universal Authorship Representations (2021.emnlp-main)
Copied to clipboard
Rafael A. Rivera-Soto, Olivia Elizabeth Miano, Juanita Ordonez, Barry Y. Chen, Aleem Khan, Marcus Bishop, Nicholas Andrews
| Challenge: | authorship verification has traditionally relied on modeling stylometric linguistic properties . but neural methods introduce a tradeoff: they obviate the need for manual feature design . |
| Approach: | They propose to use domain-specific features to improve authorship representations . they propose to study Amazon reviews, fanfiction short stories, and Reddit comments . |
| Outcome: | The proposed methods outperform existing methods in large-scale authorship verification scenarios. |
Low-Resource Fast Text Classification Based on Intra-Class and Inter-Class Distance Calculation (2025.coling-main)
Copied to clipboard
| Challenge: | Existing methods based on neural networks and pre-trained models consume substantial memory for training and text-graph construction. Existing models require access to the test dataset during the training phase, which means that when encountering new text data, the existing model needs to be retrained. |
| Approach: | They propose a low-resource and fast text classification model called LFTC to address these challenges by mining regularity information within intra-class data. |
| Outcome: | The proposed model improves performance and processing time under limited computational and data resources on 9 publicly available datasets. |
More Data, More Relations, More Context and More Openness: A Review and Outlook for Relation Extraction (2020.aacl-main)
Copied to clipboard
Xu Han, Tianyu Gao, Yankai Lin, Hao Peng, Yaoliang Yang, Chaojun Xiao, Zhiyuan Liu, Peng Li, Jie Zhou, Maosong Sun
| Challenge: | Existing methods for extracting relational facts from text have been successful . but with explosion of Web text, human knowledge is increasing drastically . |
| Approach: | They propose to improve relation extraction methods to extract relational facts from text . they analyze existing methods and show promising directions towards more powerful RE . |
| Outcome: | The proposed methods can extract relational facts from text, but they are still lacking in the current field. |
Plan-then-Generate: Controlled Data-to-Text Generation via Planning (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies focus on producing results that are close to the references, i.e. what to generate and in what order (the output structure) cannot be explicitly controlled by the users. |
| Approach: | They propose a Plan-then-Generate framework to improve the controllability of neural data-to-text models. |
| Outcome: | The proposed model can control both the intra-sentence and inter-sentent structure of the generated output. |
Promoting Graph Awareness in Linearized Graph-to-Text Generation (2021.findings-acl)
Copied to clipboard
| Challenge: | Recent applications of pretrained transformers to linearizations of graph inputs yield stateof-the-art results on graph-to-text tasks. |
| Approach: | They propose to use pretrained transformers to encode local graph structures . they find they can improve the quality of models' implicit graph encodings . |
| Outcome: | The proposed models can encode local graph structures and reconstruct corrupted inputs. |
Multimodal fusion via cortical network inspired losses (2022.acl-long)
Copied to clipboard
| Challenge: | Recent work in deep fusion models has led to substantial improvements over unimodal approaches in areas like speech recognition, emotion recognition and analysis. |
| Approach: | They propose to introduce neural dependencies into the loss functions to allow for fusion of different modalities while keeping the model complexity manageable. |
| Outcome: | Experiments on multimodal sentiment analysis tasks show that the proposed approach provides a consistent performance boost. |
Speeding Up Entmax (2022.findings-naacl)
Copied to clipboard
| Challenge: | Recent studies suggest that sparsity is a problem when the trained model is used for inference. |
| Approach: | They propose an alternative to softmax that produces a dense probability distribution but is slower than softmax. |
| Outcome: | The proposed method keeps its virtuous characteristics but is slower than softmax and achieves on par or better performance in machine translation task. |
Robust Multilingual Part-of-Speech Tagging via Adversarial Training (N18-1)
Copied to clipboard
| Challenge: | Adversarial training (AT) is a powerful regularization method for neural networks, aiming to achieve robustness to input perturbations. |
| Approach: | They propose and analyze a neural POS tagging model that exploits adversarial training by training on unmodified and adversarials. |
| Outcome: | The proposed model improves overall tagging accuracy and prevents over-fitting in low resource languages and boosts tabbing accuracy for rare / unseen words. |
Style-News: Incorporating Stylized News Generation and Adversarial Verification for Neural Fake News Detection (2024.eacl-long)
Copied to clipboard
| Challenge: | Using generative models, the issues of producing hallucinatory contents have been raised in various domains, e.g., law, writing. |
| Approach: | They propose a style-aware neural news generator that mimics the style of real news to deceive people by identifying which publisher the style corresponds to and training a model to detect fake news. |
| Outcome: | The proposed framework outperforms state-of-the-art models in terms of fluency, content preservation, and style adherence. |
Semi-Supervised Semantic Role Labeling with Cross-View Training (D19-1)
Copied to clipboard
| Challenge: | Recent approaches rely on expensive annotations and are unavailable in low resource scenarios (e.g., rare languages or domains). |
| Approach: | They propose an end-to-end SRL model which leverages unlabeled data and propose to reduce the annotation effort involved via semi-supervised learning. |
| Outcome: | The proposed model outperforms the state-of-the-art in English and consistently improves performance in other languages, including Chinese, German, and Spanish. |
Knowledge is a Region in Weight Space for Fine-tuned Language Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Specifically, language models that have been finetuned on the same dataset form a tight cluster in the weight space, while models finetuning on different datasets from the same underlying task form s looser clusters. |
| Approach: | They analyze the weight space and the underlying loss landscape of different models to determine how they are interconnected and how they perform. |
| Outcome: | The proposed method improves accuracy on 11 out of 12 datasets by starting from the center of the region, if not more, than using the pretrained model. |
Learning Logic Rules for Document-Level Relation Extraction (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing models for document-level relation extraction relied on implicitly powerful representations, which makes the model less transparent. |
| Approach: | They propose a probabilistic model for document-level relation extraction by learning logic rules. |
| Outcome: | The proposed model outperforms baseline models in relation performance and logical consistency. |
Will it Blend? Blending Weak and Strong Labeled Data in a Neural Network for Argumentation Mining (P18-2)
Copied to clipboard
Eyal Shnarch, Carlos Alzate, Lena Dankin, Martin Gleize, Yufang Hou, Leshem Choshen, Ranit Aharonov, Noam Slonim
| Challenge: | Obtaining high quality labeled data for natural language understanding tasks is slow, error-prone, complicated and expensive. |
| Approach: | They propose a method to blend weak and strong labeled data during the training of neural networks using a topic-dependent evidence detection dataset. |
| Outcome: | The proposed method improves the training of neural networks when a small amount of labeled data is available. |
Strong Baselines for Neural Semi-Supervised Learning under Domain Shift (P18-1)
Copied to clipboard
| Challenge: | Existing neural models for learning under domain shifts only evaluate on a single task, on proprietary datasets, or compare to weak baselines. |
| Approach: | They propose a multi-task tri-training method that reduces time and space complexity of classic bootstrapping approaches. |
| Outcome: | The proposed method outperforms the state-of-the-art for sentiment analysis on two benchmarks. |
Feature Drift: How Fine-Tuning Repurposes Representations in LLMs (2026.findings-eacl)
Copied to clipboard
| Challenge: | Sparse autoencoders (SAEs) are a powerful tool for interpreting neural networks by extracting concepts (features) represented in their activations. |
| Approach: | They propose to use Sparse Autoencoders to extract concepts from their activations to explain how fine-tuning changes model capabilities. |
| Outcome: | The proposed model recombines existing concepts rather than learning new ones, and shows that it is a better explanation for how fine-tuning changes model capabilities. |
DTCA: Decision Tree-based Co-Attention Networks for Explainable Claim Verification (2020.acl-main)
Copied to clipboard
| Challenge: | Recent methods to discover evidence for explainable claim verification are nontransparent and unexplained. |
| Approach: | They propose a Decision Tree-based Co-Attention model to discover evidence for explainable claim verification using neural networks. |
| Outcome: | The proposed model boosts the F1-score by more than 3.11%, 2.41% on two public datasets. |
Exploring End-to-End Differentiable Natural Logic Modeling (2020.coling-main)
Copied to clipboard
| Challenge: | Existing approaches to integrate natural logic with neural networks are brittle and prone to fail in the presence of noise and uncertainty. |
| Approach: | They propose to integrate natural logic with neural networks to create differentiable models that integrate natural reasoning with subsymbolic vector representations and neural components. |
| Outcome: | The proposed model can model monotonicity-based reasoning, compared to baseline models without inductive bias. |
Strong Baselines for Complex Word Identification across Multiple Languages (N19-1)
Copied to clipboard
Pierre Finnimore, Elisabeth Fritzsch, Daniel King, Alison Sneyd, Aneeq Ur Rehman, Fernando Alva-Manchego, Andreas Vlachos
| Challenge: | Complex Word Identification (CWI) is the task of identifying which words or phrases in a sentence are difficult to understand by a specific type of reader. |
| Approach: | They propose to use monolingual and cross-lingual CWI models to make predictions for languages not seen during training. |
| Outcome: | The proposed models perform as well as (or better than) most models submitted to the latest CWI Shared Task. |
Real-World Compositional Generalization with Disentangled Sequence-to-Sequence Learning (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to compositional generalization have been designed with semantic parsing in mind. |
| Approach: | They propose a disentangled sequence-to-sequence model which encourages more disentanglement and improves its compute and memory efficiency. |
| Outcome: | The proposed model improves generalization performance across existing tasks and datasets and a new machine translation benchmark. |
LiViTo: Linguistic and Visual Features Tool for Assisted Analysis of Historic Manuscripts (2020.lrec-1)
Copied to clipboard
| Challenge: | a mixed methods approach is feasible for the identification of scribes and authors in handwritten documents. |
| Approach: | They propose a mixed methods approach to the identification of scribes and authors in handwritten documents . they use a software tool which combines linguistic insights and computer vision techniques . |
| Outcome: | The proposed tool can be used to identify scribes and authors in handwritten documents. |
Select, Extract and Generate: Neural Keyphrase Generation with Layer-wise Coverage Attention (2021.acl-long)
Copied to clipboard
| Challenge: | Generally, documents are truncated before being inputs to deep neural networks, resulting in missing keyphrases . evaluators use layer-wise coverage attention to cover all the critical points in a document . |
| Approach: | They propose a neural keyphrase generation model that identifies the salient sentences in a document and an extractor-generator that jointly extracts and generates keyphrases from the selected sentences. |
| Outcome: | The proposed model outperforms the state-of-the-art keyphrase generation methods on keyphrases generated from scientific and web documents. |
AdaPT: A Set of Guidelines for Hyperbolic Multimodal Multilingual NLP (2024.findings-naacl)
Copied to clipboard
| Challenge: | Euclidean space is used for training neural models and performing arithmetic operations, but many data types have complex geometries and cannot be captured in the Euclidesan space. |
| Approach: | They propose a set of guidelines for initialization, parametrization, and training of neural networks that can be generalized over existing neural network training methodologies. |
| Outcome: | The proposed framework outperforms Euclidean methods on three tasks over 12 languages and modalities on a variety of domains. |
BERT-based Cohesion Analysis of Japanese Texts (2020.coling-main)
Copied to clipboard
| Challenge: | Recent advances in neural networks have significantly improved natural language processing tasks . they include self training-based language models such as BERT . |
| Approach: | They tackle a systematic analysis of cohesion in Japanese texts using BERT models . they find that coreference resolution is different in nature from other tasks . |
| Outcome: | The proposed analysis outperforms existing studies on cohesion in Japanese texts. |
Larger-Context Tagging: When and Why Does It Work? (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing tagging systems that use sentence-level data are not well understood. |
| Approach: | They propose a larger-context approach to tagging tasks that incorporates contextual information into existing tapping systems. |
| Outcome: | The proposed aggregators improve on four tagging tasks and 13 datasets. |
Quantum-inspired Language Model with Lindblad Master Equation and Interference Measurement for Sentiment Analysis (2024.naacl-long)
Copied to clipboard
| Challenge: | Quantum-inspired models have demonstrated superior performance in many downstream language tasks, such as question answering and sentiment analysis. |
| Approach: | They propose a quantum-inspired neural network that integrates the Lindblad Master Equation to model the evolution process and the interferometry to the measurement process, providing more physical meaning to strengthen the interpretability. |
| Outcome: | The proposed model outperforms existing models on sentiment analysis datasets and shows that it is more accurate and performs better than existing models. |
Competence-based Curriculum Learning for Neural Machine Translation (N19-1)
Copied to clipboard
| Challenge: | Existing NMT systems require specialized heuristics and large batch sizes. |
| Approach: | They propose a curriculum learning framework for NMT that reduces training time and costs . framework consists of a principled way of deciding which training samples are shown to the model . |
| Outcome: | The proposed framework can reduce training time and improve performance of recurrent neural network models and Transformers. |
On the Helpfulness of Document Context to Sentence Simplification (2020.coling-main)
Copied to clipboard
| Challenge: | Text simplification is a hot issue in the field of natural language generation (NLG). |
| Approach: | They propose to use Wikipedia context to improve sentence simplification by using neural networks to learn the effects of preceding and following sentences on current sentences. |
| Outcome: | The proposed model outperforms the best performing model on the baseline dataset by 2.46 (7.22%). |
Achilles-Bench: A Challenging Benchmark for Low-Resource Evaluation (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing low-resource datasets that challenge neural networks cause over-estimated performance, despite promising yet saturated results in high-res settings. |
| Approach: | They propose a benchmark Achilles-Bench to better evaluate the learning ability of neural networks in low-resource settings. |
| Outcome: | The proposed benchmarks show that even pre-trained language models show performance drops on NLP tasks. |
A Comprehensive Comparison of Neural Networks as Cognitive Models of Inflection (2022.emnlp-main)
Copied to clipboard
| Challenge: | Neural networks are at the center of a debate about human behavior in inflectional morphology. |
| Approach: | They measure correlation between human judgments and neural network probabilities for unknown word inflections. |
| Outcome: | The proposed model for morphological inflections correlates best with human wug ratings, but not with humans. |
Improving Neural RST Parsing Model with Silver Agreement Subtrees (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for Rhetorical Structure Theory (RST) parsing use supervised learning, but the RST-DT is small due to the costly annotation of RST trees. |
| Approach: | They propose to use silver data to improve RST parsing models by using annotated silver data. |
| Outcome: | The proposed method achieves the best micro-F1 scores for Nuclearity and Relation at 75.0 and 63.2 . it also achieves a remarkable gain in relation score against the previous state-of-the-art parser. |
An Augmented Benchmark Dataset for Geometric Question Answering through Dual Parallel Text Encoding (2022.coling-1)
Copied to clipboard
| Challenge: | Existing methods for solving geometric problems are limited due to lack of high-quality datasets and efficient neural solvers. |
| Approach: | They propose to annotate 2,518 geometric problems with richer types and greater difficulty using a benchmark dataset. |
| Outcome: | The proposed method improves the accuracy of automatic geometric problem solving to 66.09%. |
Norm of Word Embedding Encodes Information Gain (2023.emnlp-main)
Copied to clipboard
| Challenge: | Distributed representations of words encode lexical semantic information, but what type of information is encoded and how? |
| Approach: | They propose to use the squared norm of static word embedding to encode the information gain conveyed by the word. |
| Outcome: | The proposed model can encode the information gain of a word in a language model or neural network. |
Exploring Data Augmentation in Neural DRS-to-Text Generation (2024.eacl-long)
Copied to clipboard
| Challenge: | Neural networks are notoriously data-hungry, resulting in ungrammatical texts . data augmentation requires a specific design for a structurally rich input format . |
| Approach: | They propose to selectively augment a training set with new data by adding and varying two specific lexical categories, i.e. proper and common nouns. |
| Outcome: | The proposed approach selectively augments a training set with new data by adding and varying two specific lexical categories, i.e. proper and common nouns. |
Effects of Parameter Norm Growth During Transformer Training: Inductive Bias from Gradient Descent (2021.emnlp-main)
Copied to clipboard
| Challenge: | Evidence is emerging that neural networks learn due to inductive bias in the training routine, typically a variant of gradient descent (GD). |
| Approach: | They propose to characterize GD as an inductive bias in transformer training . they document norm growth in transformer language models and show they are saturated . |
| Outcome: | Empirically, we document norm growth in the training of transformer language models . the results suggest saturation is a new characterization of an inductive bias implicit in GD . |
Detecting cognitive impairments by agreeing on interpretations of linguistic features (N19-1)
Copied to clipboard
| Challenge: | Linguistic features have shown promising applications for detecting cognitive impairments. |
| Approach: | They propose a framework to classify after reaching agreements between modalities by using linguistic features to divide linguistic subsets into subset and let neural networks learn low-dimensional representations that agree with each other. |
| Outcome: | The proposed framework outperforms existing classifiers using all of the 413 linguistic features. |
Everything Has a Cause: Leveraging Causal Inference in Legal Text Analysis (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing studies focus on analyzing structured data, while mining causal relationship among factors from unstructured data is of great importance. |
| Approach: | They propose a graph-based causal inference framework which builds causal graphs from fact descriptions without much human involvement. |
| Outcome: | The proposed framework can capture nuance from fact descriptions among confusing charges and provide explainable discrimination in few-shot settings. |
Negative Sampling Techniques in Dense Retrieval: A Survey (2026.findings-eacl)
Copied to clipboard
| Challenge: | Information Retrieval (IR) is fundamental to many modern NLP applications. |
| Approach: | They propose a taxonomy that categorizes negative sampling techniques in dense IR . they analyze them with respect to trade-offs between effectiveness, computational cost, implementation difficulty . |
| Outcome: | The proposed taxonomy categorizes techniques using random, static/dynamically mined, and synthetic datasets. |
Stochastic Tokenization with a Language Model for Neural Text Classification (P19-1)
Copied to clipboard
| Challenge: | Sentences segmented with words or subwords can be difficult to perform text classification tasks. |
| Approach: | They propose a method to learn tokenization and text classification simultaneously to address these problems. |
| Outcome: | The proposed method improves on sentiment analysis in Japanese and Chinese using tokenization and text classification models. |
You Don’t Have Time to Read This: An Exploration of Document Reading Time Prediction (2020.acl-main)
Copied to clipboard
Orion Weller, Jordan Hildebrandt, Ilya Reznik, Christopher Challis, E. Shannon Tass, Quinn Snell, Kevin Seppi
| Challenge: | Existing work on reading time prediction has focused on word level only predictions . however, previous work has focused only on word levels . |
| Approach: | They perform an experiment to examine how different features of text contribute to the time it takes to read, distributing and collecting data from over a thousand participants. |
| Outcome: | The proposed method combines a large number of machine learning methods with textual and stylistic factors to predict the time it takes to read. |
Continual Learning Long Short Term Memory (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to prevent catastrophic forgetting in neural networks are based on the stability-plasticity dilemma, but only a limited size of old data is available. |
| Approach: | They propose a Continual Learning Long Short Term Memory cell in Recurrent Neural Network (RNN) that considers the state of each individual task's output gates and the correlation of the states between tasks. |
| Outcome: | The proposed method significantly improves on spoken language understanding tasks over state-of-the-art approaches. |
Best of Both Worlds: A Pliable and Generalizable Neuro-Symbolic Approach for Relation Classification (2024.findings-naacl)
Copied to clipboard
| Challenge: | a novel neuro-symbolic architecture for relation classification combines rule-based methods with deep learning techniques. |
| Approach: | They propose a neuro-symbolic architecture for relation classification that combines rule-based methods with deep learning techniques. |
| Outcome: | The proposed approach outperforms state-of-the-art models in three out of four settings . human interventions boost the performance on the relation org:parents by as much as 26% relative improvement . |
Leveraging Gloss Knowledge in Neural Word Sense Disambiguation by Hierarchical Co-Attention (D18-1)
Copied to clipboard
| Challenge: | Existing models for Word Sense Disambiguation use labeled data, but lack gloss knowledge. |
| Approach: | They propose a co-attention mechanism to generate co-dependent representations for context and gloss . they propose to incorporate gloss knowledge into neural networks for Word Sense Disambiguation . |
| Outcome: | The proposed model achieves state-of-the-art results on standard English all-words WSD datasets. |
An Efficient Task-Oriented Dialogue Policy: Evolutionary Reinforcement Learning Injected by Elite Individuals (2025.acl-long)
Copied to clipboard
| Challenge: | Evolutionary Algorithms (EAs) have been proven to effectively explore the solution space of neural networks by maintaining population diversity. |
| Approach: | They propose an elite individual injection mechanism to enhance EA’s search efficiency by adaptively introducing best-performing individuals into the population. |
| Outcome: | Experiments on four datasets show that the proposed approach significantly improves the balance between exploration and exploitation, boosting performance. |
An Evaluation of Progressive Neural Networksfor Transfer Learning in Natural Language Processing (2020.lrec-1)
Copied to clipboard
Abdul Moeed, Gerhard Hagerer, Sumit Dugar, Sarthak Gupta, Mainak Ghosh, Hannah Danner, Oliver Mitevski, Andreas Nawroth, Georg Groh
| Challenge: | Fine-tuning suffers from catastrophic forgetting, a problem exacerbated in natural language processing (NLP). |
| Approach: | They propose to use progressive neural networks to re-use previously learned knowledge when learning new tasks. |
| Outcome: | The proposed approach improves on common NLP tasks across a range of architectures, datasets, and tasks. |
Backpropagating through Structured Argmax using a SPIGOT (P18-1)
Copied to clipboard
| Challenge: | Structured projection of intermediate gradients (SPIGOT) is a new method for backpropagating through neural networks . structure-based learning methods for natural language processing are increasingly dominated by end-to-end differentiable functions . |
| Approach: | They propose a structured projection of intermediate gradients method for backpropagating through neural networks that includes hard-decision structured predictions in intermediate layers. |
| Outcome: | The proposed method improves on two structured NLP pipelines: syntactic-then-semantic dependency parsing and semantic parser followed by sentiment classification. |
Exploring How Generative Adversarial Networks Learn Phonological Representations (2023.acl-long)
Copied to clipboard
| Challenge: | Recent studies in natural language processing (NLP) have demonstrated two generic trends: neural networks dominate language-specific machine learning models; the interpretability of these models is limited that the language representation they learned might not align to human language. |
| Approach: | They propose to use a phonological feature-learning architecture to encode contrastive and non-contrastive nasality in French and English vowels. |
| Outcome: | The proposed architecture encodes contrastive and non-contrastive nasality in French and English vowels. |
Dynamic Meta-Embeddings for Improved Sentence Representations (D18-1)
Copied to clipboard
| Challenge: | A sprawling literature has emerged about what word embeddings are most useful for which tasks . word embed-ding is a technique that can be used to learn word-level meaning representations for a variety of tasks. |
| Approach: | They propose a method for supervised learning of embedding ensembles that leads to state-of-the-art performance on a variety of tasks. |
| Outcome: | The proposed method leads to state-of-the-art performance on a variety of tasks. |
Farewell Freebase: Migrating the SimpleQuestions Dataset to DBpedia (C18-1)
Copied to clipboard
| Challenge: | Existing datasets for question answering over knowledge graphs lack answer triples from Freebase . a defunct knowledge graph makes it difficult to build "real-world" question answering systems . |
| Approach: | They propose a benchmark dataset for simple question answering over knowledge graphs that maps SimpleQuestions entities and predicates from Freebase to DBpedia. |
| Outcome: | The proposed dataset provides simple yet strong baselines with and without neural networks. |
Unsupervised Conversation Disentanglement through Co-Training (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing work on conversation disentanglement relies heavily on human annotations, which is expensive to obtain in practice. |
| Approach: | They propose to train a conversation disentanglement model without referencing human annotations . they use a message-pair classifier and a session classifier to retrieve local relations . |
| Outcome: | The proposed method achieves competitive performance compared to previous methods on a large movie dialogue dataset. |
A Survey on Recent Advances in Named Entity Recognition from Deep Learning models (C18-1)
Copied to clipboard
| Challenge: | Named Entity Recognition (NER) is a key component in NLP systems for question answering, information retrieval, relation extraction, etc. |
| Approach: | They propose to use recurrent neural networks to generate NERs over characters, sub-words and/or word embeddings to improve named entity recognition. |
| Outcome: | The proposed architectures are better than those based on feature engineering and other supervised or semi-supervised learning algorithms. |
Data Expansion Using WordNet-based Semantic Expansion and Word Disambiguation for Cyberbullying Detection (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing methods to identify cyberbullying from text are limited due to the complexity of the content and the lack of labeled large-scale corpus. |
| Approach: | They propose a data augmentation-based approach that could enhance the automatic detection of cyberbullying in social media texts. |
| Outcome: | The proposed approach overcomes limitations of social media posts with word sense disambiguation and synonymy relation . results show that the proposed approach improves on the existing classifiers with and without data augmentation. |
Attending via both Fine-tuning and Compressing (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing studies show that attention mechanisms can improve models' interpretation, but they are not explicable. |
| Approach: | They propose a framework consisting of a learner and a compressor to purify attention scores . they propose to fine-tune and compress the attention mechanism to obtain a more faithful explanation . |
| Outcome: | The proposed framework improves performance and interpretability on eight benchmark datasets. |
Representations and Architectures in Neural Sentiment Analysis for Morphologically Rich Languages: A Case Study from Modern Hebrew (C18-1)
Copied to clipboard
| Challenge: | Existing sentiment analyzers for MRLs that use tokens and morpheme-based representations have no empirically studied effects of representation choices on neural sentiment analysis. |
| Approach: | They develop a sentiment analysis benchmark for Hebrew based on 12K social media comments and provide two instances of data. |
| Outcome: | The proposed benchmarks show that representation choices have measurable effects on task perfromance and that they vary depending on architecture type. |
Improving Deep Embedded Clustering via Learning Cluster-level Representations (2022.coling-1)
Copied to clipboard
| Challenge: | Existing efforts to learn meaningful representations at the instance level are limited. |
| Approach: | They propose a deep embedded clustering model with cluster-level representation learning to jointly learn cluster and instance level representations. |
| Outcome: | The proposed model produces meaningful clusters on real-world short text datasets. |
Towards Robustness of Text-to-SQL Models against Synonym Substitution (2021.acl-long)
Copied to clipboard
Yujian Gan, Xinyun Chen, Qiuping Huang, Matthew Purver, John R. Woodward, Jinxia Xie, Pengsheng Huang
| Challenge: | Existing text-to-SQL models rely on lexical matching between words in NL questions and tokens in table schemas, which may break the schema linking mechanism. |
| Approach: | They propose a human-curated dataset for text-to-SQL translation . they replace schema-related words with manually selected synonyms . |
| Outcome: | The proposed model outperforms its counterparts without the defense. |
Multimodal Multi-loss Fusion Network for Sentiment Analysis (2024.naacl-long)
Copied to clipboard
| Challenge: | This paper examines the optimal selection and fusion of feature encoders across multiple modalities and combines them in one neural network to improve sentiment detection. |
| Approach: | They propose to combine feature encoders across multiple modalities into one neural network to improve sentiment detection. |
| Outcome: | The proposed model achieves state-of-the-art performance for three datasets . it also shows that integrating context significantly improves model performance. |
DiLM: Distilling Dataset into Language Model for Text-level Dataset Distillation (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods to extract word embeddings from training datasets are not efficient for training other models. |
| Approach: | They propose a method to distill a training dataset into a textual model by combining a small number of informative synthetic samples. |
| Outcome: | The proposed method outperforms existing methods on training datasets and language models. |
Investigating the effect of auxiliary objectives for the automated grading of learner English speech transcriptions (2020.acl-main)
Copied to clipboard
| Challenge: | a growing demand for the ability to communicate in English means automated tutoring and assessment systems are becoming more popular. |
| Approach: | They propose to use automatic speech recognition transcripts to grade spontaneous speech based on textual features. |
| Outcome: | The proposed system improves on a transformer encoder with native language identification as an auxiliary task. |
Superpose Task-specific Features for Model Merging (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for model merging are limited by resource demands . recent studies validate the linear representation hypothesis . |
| Approach: | They propose a method that superposes task-specific features from individual models into a merged model. |
| Outcome: | The proposed method outperforms existing methods on multiple benchmarks and models. |
Incremental Beam Manipulation for Natural Language Generation (2021.eacl-main)
Copied to clipboard
| Challenge: | a larger beam size can lead to deteriorating performance of natural language generation systems due to model errors . performance of NLG systems can plateau or even decrease when beam sizes larger than 10 are used . |
| Approach: | They propose to rerank the output of beam search to produce a good set of hypotheses . they propose incremental beam manipulation to discarded hypothese . |
| Outcome: | The proposed method outperforms a strong reranker on the E2E and WebNLG datasets while being on par with the existing method. |
On the Importance of Distinguishing Word Meaning Representations: A Case Study on Reverse Dictionary Mapping (N19-1)
Copied to clipboard
| Challenge: | Sense representations target meaning conflation deficiency but their potential impact has not been investigated in downstream NLP applications. |
| Approach: | They propose to use a reverse dictionary system to address meaning conflation deficiency . they propose to integrate senses into the system to improve semantic understanding . |
| Outcome: | The proposed approach can improve the performance of a downstream NLP application. |
Continual Contrastive Spoken Language Understanding (2024.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in speech processing require extensive offline training . however, these models struggle to retain their previously acquired knowledge when learning new tasks continuously. |
| Approach: | They propose a method that relies on experience replay and contrastive learning to preserve the learned representations by pulling closer samples from the same class and pushing away the others. |
| Outcome: | The proposed method preserves the learned representations by pulling closer samples from the same class and pushing away the others. |
Automatic Poetry Generation from Prosaic Text (2020.acl-main)
Copied to clipboard
| Challenge: | In recent years, successful approaches have emerged to accurately model various aspects of natural language. |
| Approach: | They propose to combine neural networks with a poetry generation system that only uses standard text as input . they use standard text to model syntactic well-formedness and topical coherence . |
| Outcome: | The proposed framework is applied to the generation of poems in English and French . it uses standard, non-poetic text and its output is constrained to confer a poetic character . |
From Alignment to Assignment: Frustratingly Simple Unsupervised Entity Alignment (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for cross-lingual entity alignment rely on lexical matching and probability reasoning, but they inherit poor interpretability and low efficiency from neural networks. |
| Approach: | They propose a simple but effective unsupervised entity alignment method without neural networks that can be used to find the equivalent entities between crosslingual KGs. |
| Outcome: | Extensive experiments show that the proposed method beats advanced supervised methods across all datasets while having high efficiency, interpretability, and stability. |
Improving Grammatical Error Correction Models with Purpose-Built Adversarial Examples (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for grammatical error correction are data-hungry and it is hard to train a seq2seq model with good performance without suf-Clean. |
| Approach: | They propose a method inspired by adversarial training to generate more meaningful and valuable training examples by continually identifying weak spots of a model and to enhance the model by gradually adding adversarials to the training set. |
| Outcome: | The proposed method improves generalization and robustness of GEC models by adding adversarial examples to the training set. |
Simple Attention-Based Representation Learning for Ranking Short Social Media Posts (N19-1)
Copied to clipboard
| Challenge: | Existing approaches to ranking short social media posts are complex and require different components to capture a multitude of relevance signals. |
| Approach: | They propose a word-level Siamese architecture with attention-based mechanisms for capturing semantic "soft" matches between query and post tokens. |
| Outcome: | The proposed model is faster and simpler than existing models and more efficient than existing approaches. |
Incorporating Glosses into Neural Word Sense Disambiguation (P18-1)
Copied to clipboard
| Challenge: | Existing neural networks for Word Sense Disambiguation rely on labeled data and lexical knowledge. |
| Approach: | They propose a gloss-augmented WSD neural network which integrates context and glosses of the target word into a unified framework. |
| Outcome: | The proposed model outperforms the state-of-the-art systems on several English all-words WSD datasets. |
Paths to Relation Extraction through Semantic Structure (2021.findings-acl)
Copied to clipboard
| Challenge: | Syntactic and semantic structure directly reflect relations expressed by the text at hand and are therefore very useful for relation extraction (RE) |
| Approach: | They propose two methods for integrating broad-coverage semantic structure into supervised RE models by encoding semantic DAGs. |
| Outcome: | The proposed methods overshadow the use of syntactic integrations in RE . they reduce UCCA into a bilexical structure and encode semantic DAG structures . |
Semantic Training Signals Promote Hierarchical Syntactic Generalization in Transformers (2024.emnlp-main)
Copied to clipboard
| Challenge: | Neural networks without hierarchical biases struggle to learn linguistic rules that come naturally to humans . et al., 2018: Transformers trained on form and meaning favor hierarchically generalization more than those trained on forms alone. |
| Approach: | They examine whether neural networks without hierarchical biases can generalize more like humans . they find that Transformers trained on form and meaning favor hierarchic generalization . |
| Outcome: | The proposed neural networks perform better on syntactic evaluations when trained on form and meaning compared to those trained on forms alone. |
Incorporating Syntax and Frame Semantics in Neural Network for Machine Reading Comprehension (2020.coling-main)
Copied to clipboard
| Challenge: | Existing methods for machine reading comprehension rely on manually defined features and are difficult to generalize to other tasks. |
| Approach: | They propose a Syntax and Frame Semantics model for Machine Reading Comprehension which takes full advantage of syntax and frame semantics to get richer text representation. |
| Outcome: | The proposed model outperforms ten state-of-the-art models on machine reading comprehension tasks. |
Mix-and-Match: Scalable Dialog Response Retrieval using Gaussian Mixture Embeddings (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches for dialog response retrieval embed the context-response pairs as points in the embedding space. |
| Approach: | They propose a scalable model that can learn complex relationships between context-response pairs . they train the models by optimizing the Kullback-Leibler divergence between the distributions induced by context-responders in the training data. |
| Outcome: | The proposed model performs better than other embedding-based approaches on public conversation data. |
Cross-Pair Text Representations for Answer Sentence Selection (D18-1)
Copied to clipboard
| Challenge: | Existing approaches to textual entailment and question answering focus on intra-pair similarity . a simple lexical matching (marked with italics) is not enough to learn a model based on intrapair Qto-A similarities. |
| Approach: | They propose to compute scalar products representing similarity between members of different pairs instead of using a single vector for each pair. |
| Outcome: | The proposed approach outperforms more complex models based on neural networks. |
Morphosyntactic Tagging with a Meta-BiLSTM Model over Context Sensitive Token Encodings (P18-1)
Copied to clipboard
| Challenge: | recurrent neural networks have produced significant advances in part-of-speech tagging accuracy . a common feature of these models is the presence of rich initial word encodings . however, word or sub-word information interacts only through subsequent recursive layers . |
| Approach: | They propose to use recurrent neural networks with sentence-level context for initial character and word-based representations. |
| Outcome: | The proposed model has the highest accuracy of all participating systems in the CoNLL 2017 task. |
Showing Your Work Doesn’t Always Work (2020.acl-main)
Copied to clipboard
| Challenge: | countless experimental papers lack empirical rigor, disregarding necessities such as statistical significance tests and computational environments. |
| Approach: | They propose to report the expected validation effectiveness of the best-tuned model with respect to the computational budget. |
| Outcome: | The proposed model favors negative errors and yields poor bootstrapped confidence intervals, the authors argue . they find that the proposed model is biased and uses error-prone assumptions . |
Finding syntax in human encephalography with beam search (P18-1)
Copied to clipboard
| Challenge: | RNNGs are generative models of (tree , string ) pairs that evaluate derivational choices . a non-syntactic neural language model yields no reliable effects . |
| Approach: | They propose to combine a probabilistic generative grammar with a parsing procedure that uses it to manage syntactic derivations as it advances from one word to the next. |
| Outcome: | The proposed model derives two amplitude effects when used against human encephalography data. |
Cold-Start and Interpretability: Turning Regular Expressions into Trainable Recurrent Neural Networks (2020.emnlp-main)
Copied to clipboard
| Challenge: | Neural networks typically need large labeled data for training and are not easily interpretable. |
| Approach: | They propose a type of recurrent neural networks that combine neural networks and regular expression rules. |
| Outcome: | The proposed recurrent neural networks outperform previous neural approaches in low- and zero-shot scenarios and remain very competitive in rich-resource settings. |
DDisCo: A Discourse Coherence Dataset for Danish (2022.lrec-1)
Copied to clipboard
| Challenge: | Discourse coherence models have been developed using randomly shuffled texts instead of highly edited and coherent data. |
| Approach: | They propose to annotate Danish Wikipedia and Reddit for discourse coherence using real-world text instead of artificially incoherent text for training and testing models. |
| Outcome: | The proposed model performs well on annotated texts from the Danish Wikipedia and Reddit dataset. |
It’s Morphin’ Time! Combating Linguistic Discrimination with Inflectional Perturbations (2020.acl-main)
Copied to clipboard
| Challenge: | Existing work on societal bias in NLP focuses on race and gender . linguistic background is a unique attribute that has been largely ignored in the field . |
| Approach: | They examine linguistic background to craft plausible adversarial examples that expose biases in popular NLP models. |
| Outcome: | The proposed model improves robustness without sacrificing performance on clean data. |
Supervised Neural Clustering via Latent Structured Output Learning: Application to Question Intents (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent work on structured prediction has produced very effective supervised clustering algorithms using linear classifiers. |
| Approach: | They propose to use latent structured prediction loss and Transformer models to approach supervised clustering. |
| Outcome: | The proposed approach outperforms the state-of-the-art in recreating intents from public question corpora. |
Estimating the influence of auxiliary tasks for multi-task learning of sequence tagging tasks (2020.acl-main)
Copied to clipboard
| Challenge: | Multitask learning and transfer learning are techniques to overcome data scarcity . finding suitable auxiliary datasets for multitask learning is a trial-and-error approach . |
| Approach: | They propose to automatically assess the similarity of sequence tagging datasets to identify beneficial auxiliary data for MTL or TL setups. |
| Outcome: | The proposed methods can compute similarity between two sequence tagging datasets . they show that the same measures correlate with the change in test score of the auxiliary dataset . |
Quevedo: Annotation and Processing of Graphical Languages (2022.lrec-1)
Copied to clipboard
| Challenge: | graphical languages use images to convey meaning, but they require specialized computational processing . graphical systems use visual features and exploit the two dimensions of the page as a fundamental feature for codifying meaning . |
| Approach: | They present a software tool for automatic processing of graphical languages . they use a command line application and library to collect and manage image datasets . |
| Outcome: | a new software tool is developed for the processing of graphical languages . the tool provides features for the collection and management of image datasets . |
Is Attention Explanation? An Introduction to the Debate (2022.acl-long)
Copied to clipboard
Adrien Bibal, Rémi Cardon, David Alfter, Rodrigo Wilkens, Xiaoou Wang, Thomas François, Patrick Watrin
| Challenge: | Attention has been used in various tasks of NLP and other fields of machine learning to increase performance and provide some explanations. |
| Approach: | They propose to use attention as an explanation for deep learning models to increase performance . they propose to apply attention weights to queries and queries based on scalar scores . |
| Outcome: | The proposed model can be used to increase performance while providing some explanations. |
Single Model Ensemble using Pseudo-Tags and Distinct Vectors (2020.acl-main)
Copied to clipboard
| Challenge: | Existing methods for model ensembles require time, memory, and management effort to perform tasks. |
| Approach: | They propose a method that replicates the effects of a model ensemble with a single model. |
| Outcome: | The proposed method emulates or outperforms a traditional model ensemble with 1/K-times fewer parameters on text classification and sequence labeling tasks. |
Deep Contextualized Word Embeddings in Transition-Based and Graph-Based Dependency Parsing - A Tale of Two Parsers Revisited (D19-1)
Copied to clipboard
| Challenge: | In recent years, dependency parsing has shifted from discrete features to neural networks and continuous representations. |
| Approach: | They propose to use deep contextualized word embeddings to pack information about global sentence structure into local feature representations to make the two approaches virtually equivalent in terms of accuracy and error profile. |
| Outcome: | The proposed model improves the accuracy and error profile of transition-based and graph-based dependency parsers on 13 languages. |
Neural Semi-Markov Conditional Random Fields for Robust Character-Based Part-of-Speech Tagging (N19-1)
Copied to clipboard
| Challenge: | Character-level models of tokens are effective at dealing with within-token noise and out-of-vocabulary words. |
| Approach: | They propose to eliminate the need for tokenizers by using a character-level semi-Markov conditional random field that uses neural networks for its character and segment representations. |
| Outcome: | The proposed model outperforms state-of-the-art part-of speech taggers on a noisy English dataset. |
Hyperbolic Capsule Networks for Multi-Label Classification (2020.acl-main)
Copied to clipboard
| Challenge: | Existing methods for classification of labels are limited by feature aggregation and encoding. |
| Approach: | They propose to use hyperbolic capsule networks to capture fine-grained label information . they also propose a new routing method to adaptively adjust capsule number during routing . |
| Outcome: | The proposed method significantly improves the performance of multi-label classification on tail labels. |
Interpretable Neural Predictions with Differentiable Binary Variables (P19-1)
Copied to clipboard
| Challenge: | Neural networks are bringing incredible performance gains on text classification tasks, but they also require interpretability. |
| Approach: | They propose a latent model that selects a rationale and a classifier that learns from the words in the rationale alone. |
| Outcome: | The proposed model can predict expected value of penalties without REINFORCE and can be directly optimised towards a pre-specified text selection rate. |
The Paradox of the Compositionality of Natural Language: A Neural Machine Translation Case Study (2022.acl-long)
Copied to clipboard
| Challenge: | Obtaining human-like performance in NLP is often argued to require compositional generalisation. |
| Approach: | They re-instantiate three compositionality tests from the literature and reformulate them for neural machine translation. |
| Outcome: | The proposed models are more compositional than models trained on more data, the authors show . they also show that some non-compositional behaviours are mistakes, whereas others reflect natural variation in data. |
Distant Supervision Relation Extraction with Intra-Bag and Inter-Bag Attentions (N19-1)
Copied to clipboard
| Challenge: | Existing methods to extract relational data generated by distant supervision generate noisy training data. |
| Approach: | They propose a neural relation extraction method to deal with noisy training data generated by distant supervision. |
| Outcome: | Experimental results show that the proposed method is more accurate than state-of-the-art methods on the New York Times dataset. |
Modelling Instance-Level Annotator Reliability for Natural Language Labelling Tasks (N19-1)
Copied to clipboard
| Challenge: | Existing models that estimate annotators' reliability only consider binary labels and multi-class labels. |
| Approach: | They propose an unsupervised model which can handle binary and multi-class labels and integrate neural networks to model the dependency between latent variables and instances. |
| Outcome: | The proposed model can handle binary and multi-class labels and can estimate reliability of annotators across instances. |
Stock Embeddings Acquired from News Articles and Price History, and an Application to Portfolio Optimization (2020.acl-main)
Copied to clipboard
| Challenge: | Recent studies have shown that news articles can be leveraged to improve price prediction. |
| Approach: | They propose a method to encode the influence of news articles through a vector representation of stocks . they use a deep learning framework to acquire the vector representation using news articles and price history . |
| Outcome: | The proposed method can be applied to other financial problems besides price prediction. |
Grapheme-to-Phoneme Conversion for Thai using Neural Regression Models (2022.naacl-main)
Copied to clipboard
| Challenge: | Grapheme-to-phoneme conversion is a task of converting grapheme sequences into phoneme sequence. |
| Approach: | They propose a Thai grapheme-to-phoneme conversion method that uses neural networks to predict the similarity between a candidate and the correct pronunciation. |
| Outcome: | The proposed method can be applied to other languages than Thai . it is comparable to encoder-decoder models in accuracy and accuracy, it shows . |
Automatic Discovery of Heterogeneous Machine Learning Pipelines: An Application to Natural Language Processing (2020.coling-main)
Copied to clipboard
| Challenge: | Existing AutoML systems use heterogeneous techniques to build pipelines that combine techniques and algorithms from different frameworks. |
| Approach: | They propose a system for automatic machine learning that uses heterogeneous techniques. |
| Outcome: | The proposed system is evaluated in diverse machine learning problems and compared with other alternatives. |
Multimodal neural pronunciation modeling for spoken languages with logographic origin (D18-1)
Copied to clipboard
| Challenge: | Graphemes of most languages encode pronunciation, though some are more explicit than others . pronunciation modeling in logographic languages requires decomposing logographs into subunits . |
| Approach: | They propose a multimodal approach to predict pronunciation of Cantonese logographic characters using neural networks. |
| Outcome: | The proposed framework improves performance by 18.1% and 25.0% respectively to unimodal and multimodal baselines. |
Integrating Tree Structures and Graph Structures with Neural Networks to Classify Discussion Discourse Acts (C18-1)
Copied to clipboard
| Challenge: | Existing models that analyze textual contents and discussion structures require understanding of textual content and discussion structure. |
| Approach: | They propose a model that integrates discussion structures with neural networks to classify discourse acts. |
| Outcome: | The proposed model improves accuracy and FB1 score by 1.5% compared to the previous best model. |
Named Entity Recognition through Deep Representation Learning and Weak Supervision (2021.findings-acl)
Copied to clipboard
| Challenge: | Weakly supervised named entity recognition (NER) uses noisy labels to estimate the true labels of a dataset. |
| Approach: | They propose a model to learn optimal assignments of latent NER tags using observed tokens and weak labels provided by labeling functions. |
| Outcome: | The proposed model improves the quality of weak labels on four public datasets. |
On the Importance of Delexicalization for Fact Verification (D19-1)
Copied to clipboard
| Challenge: | Neural networks (NNs) perform state-of-the-art (SOA) performance in many complex tasks. |
| Approach: | They investigate the importance that a model assigns to various aspects of data . they experiment with two strategies of masking to mitigate this dependence on lexicalized information . |
| Outcome: | The proposed model improves on the in-domain dataset by 10% compared to the fully lexicalized model. |
Multi-Task Learning for Japanese Predicate Argument Structure Analysis (N19-1)
Copied to clipboard
| Challenge: | Recent work ignores event-nouns or builds a single model for solving both tasks . however, there are interactions between predicates and event-nons, making it difficult to target only predicate. |
| Approach: | They propose a multi-task learning method that targets event-nouns . their results improve performance of both PASA and ENASA tasks . |
| Outcome: | The proposed model improves both PASA and ENASA tasks compared to a single-task model . it is the first work to employ neural networks in ENASA . |
ContraSim – Analyzing Neural Representations Based on Contrastive Learning (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing similarity measures perform mediocrely on standard benchmarks . |
| Approach: | They develop a similarity measure based on contrastive learning that learns a parameterized measure by using both similar and dissimilar examples. |
| Outcome: | The proposed measure achieves much higher accuracy than previous similarity measures . it is more suitable for the analysis of neural networks, revealing new insights . |
Learning Implicit Text Generation via Feature Matching (2020.acl-main)
Copied to clipboard
Inkit Padhi, Pierre Dognin, Ke Bai, Cícero Nogueira dos Santos, Vijil Chenthamarakshan, Youssef Mroueh, Payel Das
| Challenge: | Generative feature matching networks are an approach for training implicit generative models for images . a novel formulation of GFMN for unconditional sequence generation is proposed . |
| Approach: | They propose a new GFMN formulation that performs token level feature matching on pre-trained neural networks. |
| Outcome: | The proposed method outperforms adversarial approaches for text generation and style transfer. |
Variational Autoregressive Decoder for Neural Response Generation (D18-1)
Copied to clipboard
| Challenge: | Existing variational Bayesian models generate responses from a single latent variable, which is not sufficient to model high variability in responses. |
| Approach: | They propose a conditional variable auto-encoder that sequentially introduces latent variables to condition the generation of each word in the response sequence. |
| Outcome: | Empirical results show that the proposed model improves on state-of-the-art models on Opensubtitle and Reddit datasets. |
Energy and Policy Considerations for Deep Learning in NLP (P19-1)
Copied to clipboard
| Challenge: | Recent advances in hardware and methodology for training neural networks have enabled significant accuracy improvements across many NLP tasks. |
| Approach: | They quantify the approximate financial and environmental costs of training neural network models . they propose actionable recommendations to reduce costs and improve equity in NLP research . |
| Outcome: | The proposed recommendations address the cost and environmental costs of training neural networks for NLP. |
GlossBERT: BERT for Word Sense Disambiguation with Gloss Knowledge (D19-1)
Copied to clipboard
| Challenge: | Word Sense Disambiguation (WSD) aims to find the exact sense of an ambiguous word in a particular context. |
| Approach: | They propose to integrate gloss knowledge into supervised neural networks for Word Sense Disambiguation (WSD) this paper proposes to fine-tune a pre-trained BERT model and achieve new state-of-the-art results on WSD task. |
| Outcome: | The proposed model achieves state-of-the-art on the word Sense Disambiguation (WSD) task. |
Data and Model Distillation as a Solution for Domain-transferable Fact Verification (2021.naacl-main)
Copied to clipboard
| Challenge: | Neural networks depend heavily on lexicalized information, which transfers poorly between domains. |
| Approach: | They propose a method to delexicize lexicalized data and a model distillation technique to prevent aggressive data distillation. |
| Outcome: | The proposed method improves performance on lexicalized data and out of domain models. |
Successfully Applying the Stabilized Lottery Ticket Hypothesis to the Transformer Architecture (2020.acl-main)
Copied to clipboard
| Challenge: | Current neural networks are heavily growing in depth, with many fully connected layers. |
| Approach: | They propose to combine stabilized lottery ticket pruning with unstructured pruning to improve model performance. |
| Outcome: | The proposed pruning techniques outperform all other techniques for even higher sparsity levels. |
We Need to Talk About train-dev-test Splits (2021.emnlp-main)
Copied to clipboard
| Challenge: | Standard train-dev-test splits used to benchmark multiple models are now used in NLP . comparing multiple versions of the same model on the test data leads to overfitting and "expiration" of test sets. |
| Approach: | They propose to use a tune-set when developing neural network methods to do model picking. |
| Outcome: | The proposed model picker is more robust against the evaluated hyperparameter ranges than the standard split split. |
A Search-based Neural Model for Biomedical Nested and Overlapping Event Detection (D19-1)
Copied to clipboard
| Challenge: | Existing structured prediction tasks target nested and overlapping events . a new structured prediction model is proposed that uses a relation graph to detect overlapping and nesting events. |
| Approach: | They propose a search-based neural network structured prediction model that treats the task as a searching problem on a relation graph of trigger-argument structures. |
| Outcome: | The proposed model performs comparable to the state-of-the-art model Turku Event Extraction System (TEES) on the BioNLP Cancer Genetics (CG) Shared Task 2013 without the use of syntactic and hand-engineered features. |
Benchmarking Neural Topic Models: An Empirical Study (2021.findings-acl)
Copied to clipboard
| Challenge: | Neural topic modeling has been attracting much attention recently due to its ability to leverage the advantages of both neural networks and probabilistic topic models. |
| Approach: | They propose to evaluate neural topic models in three tasks using large datasets and a set of metrics to compare them. |
| Outcome: | The proposed models perform better in the first and third tasks than the traditional probabilistic models and are better in many cases. |
Fully Hyperbolic Neural Networks (2022.acl-long)
Copied to clipboard
| Challenge: | Existing hyperbolic neural networks encode features in the hyperbolical space yet formalize most of their operations in the tangent space. |
| Approach: | They propose a fully hyperbolic framework to build hyperbolical networks based on the Lorentz model by adapting Lorentzer transformations to formalize essential operations of neural networks. |
| Outcome: | The proposed framework has better performance on four NLP tasks compared with existing hyperbolic models . |
A Multilingual Evaluation Dataset for Monolingual Word Sense Alignment (2020.lrec-1)
Copied to clipboard
Sina Ahmadi, John Philip McCrae, Sanni Nimb, Fahad Khan, Monica Monachini, Bolette Pedersen, Thierry Declerck, Tanja Wissik, Andrea Bellandi, Irene Pisani, Thomas Troelsgård, Sussi Olsen, Simon Krek, Veronika Lipp, Tamás Váradi, László Simon, András Gyorffy, Carole Tiberius, Tanneke Schoonheim, Yifat Ben Moshe, Maya Rudich, Raya Abu Ahmad, Dorielle Lonke, Kira Kovalenko, Margit Langemets, Jelena Kallas, Oksana Dereza, Theodorus Fransen, David Cillessen, David Lindemann, Mikel Alonso, Ana Salgado, José Luis Sancho, Rafael-J. Ureña-Ruiz, Jordi Porta Zamorano, Kiril Simov, Petya Osenova, Zara Kancheva, Ivaylo Radev, Ranka Stanković, Andrej Perdih, Dejan Gabrovsek
| Challenge: | a new dataset aims to align monolingual dictionaries with a single sense level for 15 languages . this dataset covers a wide range of languages and resources . |
| Approach: | They propose to manually align monolingual dictionaries with possible semantic relationships . they use 15 languages to create a new baseline for the task of monolingual word sense alignment . |
| Outcome: | The proposed dataset covers 15 languages and covers the more challenging task of linking general-purpose language. |
Enhancing Language Model Hypernetworks with Restart: A Study on Optimization (2025.naacl-long)
Copied to clipboard
| Challenge: | a comprehensive investigation into optimization strategies for hypernetworks remains lacking. |
| Approach: | They propose restart optimization strategies to improve hypernetworks' performance for language models. |
| Outcome: | The proposed restart strategy improves hypernetworks' performance for language models, compared to conventional deep neural networks. |
Data-Efficient Language Shaped Few-shot Image Classification (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies have shown that language is helpful guider for image understanding by neural networks. |
| Approach: | They propose a language-shaped learning method that makes the best use of the few-shot images and the language available only in training. |
| Outcome: | The proposed method outperforms state-of-the-art methods on a few-shot dataset with limited training data. |
Improving a Neural-based Tagger for Multiword Expressions Identification (L18-1)
Copied to clipboard
| Challenge: | MUMULS tagger for automatic detection of verbal multiword expressions is based on neural networks . character-level embeddings can improve the performance, reducing out-of-vocabulary rate . multiword Expressions are viewed by computational linguists as a "pain in the neck of NLP" |
| Approach: | They propose to improve MUMULS, a tagger for automatic detection of verbal multiword expressions. |
| Outcome: | The proposed tagger performed better on Czech language than the previous taggers. |
ProSeqo: Projection Sequence Networks for On-Device Text Classification (D19-1)
Copied to clipboard
| Challenge: | ProSeqo is a novel on-device sequence model for text classification . it uses dynamic recurrent projections without the need to store or look up pre-trained embeddings. |
| Approach: | They propose a novel on-device sequence model for text classification using recurrent projections that uses dynamic recursion projections without the need to store or look up any pre-trained embeddings. |
| Outcome: | The proposed model outperforms state-of-the-art neural and on-device approaches for short and long text classification tasks while maintaining low memory footprint and high accuracy. |
Improving Hate Speech Detection with Deep Learning Ensembles (L18-1)
Copied to clipboard
| Challenge: | censorship is a potential risk when addressing these issues with automated text classification methods. |
| Approach: | They propose to use a neural network-based ensemble method to better classify hate speech using a publicly available embedding model and a popular sentiment dataset. |
| Outcome: | The proposed method improves by 5 points on a hate speech corpus from Twitter and a popular sentiment dataset. |
Training for Gibbs Sampling on Conditional Random Fields with Neural Scoring Factors (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in NLP focus on simple approaches to model the output label space . graphical models are often limited to (heuristic) greedy search and its variants . |
| Approach: | They propose an approach for efficiently training and decoding hybrids of graphical and graphical models based on Gibbs sampling. |
| Outcome: | The proposed approach improves on Dutch and Dutch with graphical models . the proposed model improves over a strong baseline on three languages . |
BanditSum: Extractive Summarization as a Contextual Bandit (D18-1)
Copied to clipboard
| Challenge: | Existing methods for extractive summarization are heuristically generated and require a set of binary labels to be selected. |
| Approach: | They propose a method for training neural networks to perform single-document extractive summarization without heuristically-generated extractive labels. |
| Outcome: | The proposed method achieves better ROUGE scores than the state-of-the-art methods and significantly fewer update steps than competing approaches. |
Analyzing Code Embeddings for Coding Clinical Narratives (2021.findings-acl)
Copied to clipboard
| Challenge: | Recent work on automated ICD coding learn mappings between low-dimensional representations of clinical text reports and codes. |
| Approach: | They propose novel neural networks for encoding medical codes based on textual, structural and statistical characteristics using a single deep learning baseline model. |
| Outcome: | The proposed methods improve the accuracy of medical codes based on their textual, structural and statistical characteristics. |
Learning Only from Relevant Keywords and Unlabeled Documents (D19-1)
Copied to clipboard
| Challenge: | Existing methods for document classification are limited due to labeling and privacy concerns. |
| Approach: | They propose a super-vised text classification framework that provides keywords as a hint for classifying a document to a target class. |
| Outcome: | The proposed framework is simple to implement and has flexible choices of models, e.g., linear models or neural networks. |
Enhancing Variational Autoencoders with Mutual Information Neural Estimation for Text Generation (D19-1)
Copied to clipboard
| Challenge: | Existing approaches to train variational autoencoders (VAEs) have been proposed to alleviate the posterior collapse issue in NLP tasks. |
| Approach: | They propose to introduce a mutual information term between the input and its latent variable to regularize the objective of the VAE. |
| Outcome: | The proposed model performs better on three benchmark datasets and is comparable to state-of-the-art models. |
Human Attention Maps for Text Classification: Do Humans and Neural Networks Focus on the Same Words? (2020.acl-main)
Copied to clipboard
| Challenge: | Attention-based models have been claimed to add interpretability, but little is known about the actual relationships between machine and human attention. |
| Approach: | They conduct the first quantitative assessment of human versus computational attention mechanisms for the text classification task. |
| Outcome: | The proposed models are compared against machine attention maps on a publicly available YELP dataset. |
Information-Theoretic Probing for Linguistic Structure (2020.acl-main)
Copied to clipboard
| Challenge: | Neural networks are the backbone of modern stateof-the-art natural language processing systems. |
| Approach: | They propose an information-theoretic operationalization of probing as estimating mutual information that contradicts received wisdom . they evaluate on a set of ten typologically diverse languages often underrepresented in NLP research—plus English—totalling eleven languages. |
| Outcome: | The proposed model outperforms existing models on ten typologically diverse languages and English on 11 languages. |
Optimizing Hidden Markov Language Models: An Empirical Study of Reparameterization and Initialization Techniques (2025.findings-naacl)
Copied to clipboard
| Challenge: | Recent work on scaling-up HMMs to perform competitively as language models has indicated that this challenge only increases with larger hidden state sizes. |
| Approach: | They propose two strategies that use neural reparameterization and neural initialization to enhance HMM optimization. |
| Outcome: | The proposed techniques work well for scaled HMM language modeling, and linear reparameterizations can be as effective as non-linear ones, and the strategies are complementary. |
Chinese Relation Extraction with Multi-Grained Information and External Linguistic Knowledge (P19-1)
Copied to clipboard
| Challenge: | Existing methods for Chinese relation extraction suffer from segmentation errors and ambiguity of polysemy. |
| Approach: | They propose a multi-grained lattice framework for Chinese relation extraction . they incorporate word-level information into character sequence inputs to avoid segmentation errors . |
| Outcome: | The proposed model outperforms existing models on three real-world datasets in distinct domains. |
Exploring Distantly-Labeled Rationales in Neural Network Models (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods focus on distantly-labeled rationales, ignoring the potential important non-rationale words and not distinguishing the importance of different rationale words. |
| Approach: | They propose two novel auxiliary loss functions to make better use of distantly-labeled rationales, which encourage models to maintain their focus on important words beyond labeled rationals (PINs) and alleviate redundant training on non-helpful rationale (NoIRs). |
| Outcome: | The proposed methods outperform existing methods on two representative classification tasks while maintaining the ability to spread focus to other unlabeled important words. |
Hate Speech and Counter Speech Detection: Conversational Context Does Matter (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing datasets and models target hate speech but ignore context . Existing models target either hate speech or hate and counter speech but disregard context - a new study shows that context is critical to identify hate and anti-hate speech. |
| Approach: | They propose to use context to identify hate and counter speech in a reddit conversation thread. |
| Outcome: | The proposed model improves when and why context is taken into account. |
FlowSeq: Non-Autoregressive Conditional Sequence Generation with Generative Flow (D19-1)
Copied to clipboard
| Challenge: | Neural sequence-to-sequence models are autoregressive, meaning they factor the joint probability of the output sequence into the product of probabilities over the next to-ken. |
| Approach: | They propose a non-autoregressive sequence generation model using latent variables . they use generative flow to model complex distributions using neural networks . |
| Outcome: | The proposed model performs comparable to state-of-the-art models and has constant decoding time w.r.t the sequence length. |
Compositional Generalization for Primitive Substitutions (D19-1)
Copied to clipboard
| Challenge: | Existing approaches to encoding compositional generalization are lacking . et al., 2017) argue that neural networks lack compositional ability . |
| Approach: | They propose a method to encode compositionality in neural networks using two representations . they reduce the entropy in each representation to improve generalization . |
| Outcome: | The proposed approach improves performance on five NLP tasks including instruction learning and machine translation. |
Prompt-Based Monte-Carlo Tree Search for Goal-oriented Dialogue Policy Planning (2023.emnlp-main)
Copied to clipboard
| Challenge: | Optimal policy planning is a difficult task, authors say . many goal-oriented conversations require subjective strategies, they say - a problem in goal-orientated settings . |
| Approach: | They propose an approach to perform goal-oriented dialogue policy planning without model training. |
| Outcome: | The proposed approach performs goal-oriented dialogue policy planning without model training. |
Automatically Select Emotion for Response via Personality-affected Emotion Transition (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing studies focus on rendering specified emotions in responses, yet the individual difference in emotion expression is overlooked. |
| Approach: | They propose to equip a dialog system with personality and enable it to select emotions in responses like humans. |
| Outcome: | The proposed system can select emotions in responses like humans by simulating the emotion transition of humans in conversation. |
Sequence-Level Mixed Sample Data Augmentation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Despite their empirical success, neural networks still have difficulty capturing compositional aspects of natural language. |
| Approach: | They propose a data augmentation approach to encourage compositional behavior in neural networks . they propose to softly combine input/output sequences from the training set . |
| Outcome: | The proposed approach yields 1.0 BLEU improvement on translation datasets over baselines. |
Ensemble Distillation for Structured Prediction: Calibrated, Accurate, Fast—Choose Three (2020.emnlp-main)
Copied to clipboard
| Challenge: | Modern neural networks do not always produce wellcalibrated predictions . post-hoc calibration methods require a held-out calibration dataset, which may not be available in all circumstances. |
| Approach: | They validate ensemble distillation framework for producing well-calibrated structured prediction models without the prohibitive inference-time cost of ensembles. |
| Outcome: | The proposed framework produces well-calibrated predictions without the prohibitive inference-time cost of ensembles. |
Dilated Convolutional Neural Networks for Lightweight Diacritics Restoration (2022.lrec-1)
Copied to clipboard
| Challenge: | Diacritics restoration is a ubiquitous task in the Latin-alphabet-based English-dominated Internet language environment. |
| Approach: | They propose a 1D dilated convolution-based approach which operates on a character-level. |
| Outcome: | The proposed approach surpasses similar models and is competitive with larger models. |
Explain by Evidence: An Explainable Memory-based Neural Network for Question Answering (2020.coling-main)
Copied to clipboard
| Challenge: | Interpretability and explainability of deep neural net models are always challenging due to their size and complexity. |
| Approach: | They propose to design an explainable, evidence-based memory network architecture that connects current sample with seen samples and bases its decision on these samples. |
| Outcome: | The proposed model can trace errors to training instances that might have caused errors . the proposed model achieves state-of-the-art performance on two popular datasets . |
GKnow: Measuring the Entanglement of Gender Bias and Factual Gender (2026.acl-long)
Copied to clipboard
| Challenge: | Recent studies have focused on mitigating gender bias, but mechanistic interpretations of gender fail to distinguish between factually gendered outputs and gender biased outputs. |
| Approach: | They propose a benchmark to assess gender knowledge and gender bias in language models . they use neuron ablation to disentangle stereotypical and factual gender . |
| Outcome: | The proposed benchmark assesses gender knowledge and gender bias in language models across different types of gender-related predictions. |
SUN: Exploring Intrinsic Uncertainties in Text-to-SQL Parsers (2022.coling-1)
Copied to clipboard
Bowen Qin, Lihan Wang, Binyuan Hui, Bowen Li, Xiangpeng Wei, Binhua Li, Fei Huang, Luo Si, Min Yang, Yongbin Li
| Challenge: | Existing methods that learn from multiple semantically-equivalent questions are limited to one-to-one mapping . |
| Approach: | They propose a constraint to explore the underlying complementary semantic information among multiple semantically-equivalent questions and learn robust feature representations with reduced spurious associations. |
| Outcome: | The proposed method outperforms strong competitors and achieves state-of-the-art results on five benchmark datasets. |
Different Absorption from the Same Sharing: Sifted Multi-task Learning for Fake News Detection (D19-1)
Copied to clipboard
| Challenge: | Existing methods for detecting fake news use shared features as complementarity features without selection. |
| Approach: | They propose a sifted multi-task learning method with a selected sharing layer for fake news detection. |
| Outcome: | The proposed method boosts the F1-score by more than 0.87%, 1.31% on two public and widely used competition datasets. |
Is it Time to Swish? Comparing Deep Learning Activation Functions Across NLP tasks (D18-1)
Copied to clipboard
| Challenge: | Activation functions are nonlinearities which have been attributed to the success story of deep learning. |
| Approach: | They propose to use a penalized tanh function to replace the sigmoid and tansh gates in LSTM cells and to improve the performance of the activation function. |
| Outcome: | The proposed activation function performs best on all tasks and can replace the sigmoid and tanh gates in LSTM cells. |
Knowledge Base Embedding By Cooperative Knowledge Distillation (2020.coling-main)
Copied to clipboard
| Challenge: | Knowledge bases are increasingly exploited as gold standard data sources for various knowledge-driven NLP tasks. |
| Approach: | They propose a method to perform knowledge base representation learning by mutually and jointly distilling knowledge within a dynamic teacher-student setting. |
| Outcome: | The proposed approach outperforms two baselines, traditional and sequential, on two standard datasets showing that it is possible to distill knowledge between KBs. |
Finding Universal Grammatical Relations in Multilingual BERT (2020.acl-main)
Copied to clipboard
| Challenge: | Recent work has found that multilingual masked language models learn a surprising amount of linguistic structure, despite a lack of direct linguistic supervision. |
| Approach: | They propose an unsupervised method to find syntactic tree distances in languages other than English and that these subspaces are approximately shared across languages. |
| Outcome: | The proposed method shows that mBERT learns representations of syntactic dependency labels, in the form of clusters, which largely agree with the Universal Dependencies taxonomy. |
Improved Transcription and Indexing of Oral History Interviews for Digital Humanities Research (L18-1)
Copied to clipboard
| Challenge: | Existing methods to improve transcription and indexing quality of Oral History interviews are not available. |
| Approach: | They propose to use a German Oral History test-set to improve transcription and indexing quality . they propose to combine acoustic modeling techniques with sophisticated neural networks . |
| Outcome: | The proposed system reduces word error rate by 28.3% on German Oral History test-set compared to baseline system . the Fraunhofer IAIS Audio Mining system can process long audio-files to automatically create time-aligned transcriptions. |
Generating Hierarchical Explanations on Text Classification via Feature Interaction Detection (2020.acl-main)
Copied to clipboard
| Challenge: | Existing methods for generating explanations for neural networks ignore feature interactions between words and phrases. |
| Approach: | They propose to build hierarchical explanations by detecting feature interactions by combining words and phrases at different levels of the hierarchy. |
| Outcome: | The proposed method is evaluated on two benchmark datasets, via automatic and human evaluations. |
Open ASR for Icelandic: Resources and a Baseline System (L18-1)
Copied to clipboard
| Challenge: | Existing language resources are not sufficient for less-resourced languages, but a system with sufficient resources is needed. |
| Approach: | They describe available language resources and their preparation for use in a large vocabulary speech recognition system for Icelandic. |
| Outcome: | The proposed system improves on acoustic training sets and a speech corpus with a pronunciation dictionary. |
HOLMS: Alternative Summary Evaluation with Large Language Models (2020.coling-main)
Copied to clipboard
| Challenge: | Efficient document summarization requires evaluation measures that can rank a set of systems based on an average score and highlight which individual summary is better than another. |
| Approach: | They propose a hybrid evaluation measure for document summarization called HOLMS that combines both language models pre-trained on large corpora and lexical similarity measures. |
| Outcome: | The proposed measure outperforms ROUGE and BLEU on several extractive summarization datasets for both linguistic quality and pyramid scores. |
Joint Learning for Targeted Sentiment Analysis (D18-1)
Copied to clipboard
| Challenge: | Recent studies have attempted to perform two tasks separately, e.g., target extraction and sentiment classification. |
| Approach: | They propose a hierarchical stack bidirectional gated recurrent units (HSBi-GRU) model which allows the target label to influence their sentiment label. |
| Outcome: | The proposed model outperforms baseline models on two datasets and shows that it can learn abstract features. |
PRADO: Projection Attention Networks for Document Classification On-Device (D19-1)
Copied to clipboard
| Challenge: | Recent advances in deep learning have improved the performance of on-device neural networks for long text classification. |
| Approach: | They propose a projection attention neural network PRADO that combines trainable projections with attention and convolutions to train tiny neural networks that achieve high performance on multiple long document classification tasks. |
| Outcome: | The proposed model achieves high performance on multiple long document classification tasks while maintaining compact size. |
Word-order Biases in Deep-agent Emergent Communication (P19-1)
Copied to clipboard
| Challenge: | a recent study examines the "natural" word-order constraints that constrain neural networks . we train models to communicate about paths in a simple gridworld . |
| Approach: | They propose to inoculate a notion of "effort" into neural networks to make their linguistic behavior more human-like. |
| Outcome: | The proposed models show a strong tendency to avoid redundancy and minimize long-distance dependencies. |
Auto-hMDS: Automatic Construction of a Large Heterogeneous Multilingual Multi-Document Summarization Corpus (L18-1)
Copied to clipboard
| Challenge: | Existing datasets for automatic text summarization are small and focused on newswires. |
| Approach: | They propose to automatically generate a large multilingual multi-document summarization corpus using Wikipedia articles as summaries and to automatically search for appropriate source documents. |
| Outcome: | The proposed corpus contains 7,316 topics in English and German with different summary lengths and number of source documents. |
Empowering Low-Resource Regional Languages with Lexicons : A Comparative Study of NLP Tools for Morphosyntactic Analysis (2024.lrec-main)
Copied to clipboard
| Challenge: | a lack of human and financial resources makes integrating lexicon information to low-resource languages challenging. |
| Approach: | They propose to use a bilingual lexicon to integrate lexical information to low-resource language . they compare a lexiconal approach to a neural approach that uses a larger lexicone . |
| Outcome: | The proposed approach improves POS tagging while using different lexicon sizes. |
Train One Sparse Autoencoder Across Multiple Sparsity Budgets to Preserve Interpretability and Accuracy (2025.emnlp-main)
Copied to clipboard
Nikita Balagansky, Yaroslav Aksenov, Daniil Laptev, Vadim Kurochkin, Gleb Gerasimov, Nikita Koriagin, Daniil Gavrilov
| Challenge: | Sparse Autoencoders (SAEs) are powerful tools for interpreting neural networks . conventional SAEs are constrained by the fixed sparsity level chosen during training . |
| Approach: | They propose a training objective that trains a single SAE to optimise reconstructions across multiple sparsity levels simultaneously. |
| Outcome: | The proposed objective achieves Pareto-optimal trade-offs between sparsity and explained variance, outperforming traditional SAEs trained at individual sparsities. |
Explainable Depression Detection in Clinical Interviews with Personalized Retrieval-Augmented Generation (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing systems rely on black-box neural networks, which lack interpretability, which is crucial in mental health contexts. |
| Approach: | They propose a Retrieval-augmented generation framework for Explainable depression detection that retrieves evidence from clinical interview transcripts, providing explanations for predictions. |
| Outcome: | The proposed framework retrieves evidence from clinical interview transcripts, providing explanations for predictions. |
How poor is the stimulus? Evaluating hierarchical generalization in neural networks trained on child-directed speech (2023.acl-long)
Copied to clipboard
| Challenge: | LSTMs and Transformers perform well at capturing the surface statistics of child-directed speech, but both model types generalize in a way consistent with an incorrect linear rule than the correct hierarchical rule. |
| Approach: | They train LSTMs and Transformers on text from the CHILDES corpus and evaluate what they learn about English yes/no questions. |
| Outcome: | The proposed models perform well at capturing the surface statistics of child-directed speech, but generalize more consistent with an incorrect linear rule than the correct hierarchical rule. |
Adversarial Reprogramming of Text Classification Neural Networks (D19-1)
Copied to clipboard
| Challenge: | Recent studies have shown that adversarial examples can cause a machine learning model to misclassify a sample from the classifier's input domain. |
| Approach: | They propose a context-based vocabulary remapping method that performs a computationally inexpensive input transformation to reprogram a victim classification model for a new set of sequences. |
| Outcome: | The proposed method performs a cost-effective input transformation to reprogram a model for a new set of sequences without altering the network architecture or parameters. |
Unveiling Fake News with Adversarial Arguments Generated by Multimodal Large Language Models (2025.coling-main)
Copied to clipboard
| Challenge: | Existing methods for detecting fake news rely on neural networks to learn latent feature representations with limited real-world understanding. |
| Approach: | They propose a method that leverages Multimodal Large Language Models for fake news detection that introduces adversarial reasoning through debates from opposing perspectives. |
| Outcome: | The proposed method significantly outperforms state-of-the-art methods on four fake news detection datasets. |
Adapting Serious Game for Fallacious Argumentation to German: Pitfalls, Insights, and Best Practices (L18-1)
Copied to clipboard
| Challenge: | 'homeschooling' and 'death penalty' are non-existent in Germany, while being highly controversial topics of discussion in the United States. |
| Approach: | They propose to port Argotario (serious game for learning argumentation fallacies) to another language and analyze users' behavior and in-game created data to assess dissemination strategies and qualitative aspects of the resulting corpus. |
| Outcome: | The proposed game is based on a German-based game platform that can be used to learn argumentation fallacies. |
Attribute Diversity Determines the Systematicity Gap in VQA (2024.emnlp-main)
Copied to clipboard
| Challenge: | a systematicity gap exists between neural networks generalizing to new combinations of familiar concepts . conventionally trained neural networks struggle to generalize systematically . |
| Approach: | They propose to train a visual question answering model with CLEVR-HOPE as a diagnostic dataset to test this hypothesis. |
| Outcome: | The systematicity gap is reduced by increasing the diversity of training data, the authors show . the authors suggest that the more distinct attribute type combinations are seen during training, the more systematic the model will be. |
Do Neural Models Learn Systematicity of Monotonicity Inference in Natural Language? (2020.acl-main)
Copied to clipboard
| Challenge: | Despite the success of language models using neural networks, it remains unclear to what extent neural models have the generalization ability to perform inferences. |
| Approach: | They propose a method to evaluate whether neural models can learn systematicity of monotonicity inference in natural language. |
| Outcome: | The proposed method shows that neural models can perform inferences on unseen combinations of lexical and logical phenomena when syntactic structures are similar between training and test sets. |
On the Impact of Calibration Data in Post-training Quantization and Pruning (2024.acl-long)
Copied to clipboard
| Challenge: | Quantization and pruning are the foundations of compression for large language models . however, no prior work has investigated how calibration data impacts performance of compression methods. |
| Approach: | They propose an empirical study on the effect of calibration data on LLM performance. |
| Outcome: | The proposed methods improve performance in a post-training setting. |
Feature Interactions Reveal Linguistic Structure in Language Models (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing features attribution methods for post-hoc interpretability ignore the existence of interactions between the effects of features on the prediction. |
| Approach: | They propose a grey box method to train models to perfection on a formal language classification task using PCFGs. |
| Outcome: | The proposed methods are able to uncover the grammatical rules acquired by the model under specific configurations and provide novel insights into the linguistic structure of the target models. |
NatLogAttack: A Framework for Attacking Natural Language Inference Models with Natural Logic (2023.acl-long)
Copied to clipboard
| Challenge: | Despite the recent advances in distributed representation and neural networks, it remains an open question whether the models perform real reasoning to reach their conclusions or rely on spurious correlations. |
| Approach: | They propose to use logic formalism to perform systematic attacks centring around natural logic to generate better adversarial examples with fewer visits to the victim models. |
| Outcome: | The proposed framework generates better adversarial examples with fewer visits to the victim models. |
LexicalAT: Lexical-Based Adversarial Reinforcement Training for Robust Sentiment Classification (D19-1)
Copied to clipboard
| Challenge: | Existing text classification models are fragile and sensitive to simple perturbations. |
| Approach: | They propose a generator-classifier adversarial training approach to improve classification models . they use a large-scale lexical knowledge base to generate attacking examples . |
| Outcome: | The proposed approach outperforms strong baselines and reduces test errors on neural networks. |
Students Who Study Together Learn Better: On the Importance of Collective Knowledge Distillation for Domain Transfer in Fact Verification (2021.emnlp-main)
Copied to clipboard
| Challenge: | Neural networks depend heavily on lexicalized information, which can be overfitted . this can be a problem in fact verification, which has important societal implications. |
| Approach: | They propose a knowledge distillation approach for fact verification using student models. |
| Outcome: | The proposed approach outperforms state-of-the-art classifiers on a training dataset and in supervised settings. |
Human-Like Decision Making: Document-level Aspect Sentiment Classification via Hierarchical Reinforcement Learning (D19-1)
Copied to clipboard
Jingjing Wang, Changlong Sun, Shoushan Li, Jiancheng Wang, Luo Si, Min Zhang, Xiaozhong Liu, Guodong Zhou
| Challenge: | Recent neural networks have shown promising results on Document-level Aspect Sentiment Classification (DASC) however, these approaches often offer little transparency w.r.t. their inner working mechanisms and lack interpretability. |
| Approach: | They propose a Hierarchical Reinforcement Learning approach to DASC that incorporates clause selection and word selection strategies to tackle the data noise problem. |
| Outcome: | The proposed approach over the state-of-the-art approaches shows impressive performance over the current baselines. |
The Unstoppable Rise of Computational Linguistics in Deep Learning (2020.acl-main)
Copied to clipboard
| Challenge: | a quarter century ago, linguists assumed that language knowledge needed to be innate . but vector-space representations and machine learning algorithms are much more powerful than was thought . |
| Approach: | They trace the history of neural networks applied to natural language understanding tasks . they argue that Transformer is not a sequence model but an induced-structure model . |
| Outcome: | The proposed model is not a sequence model but an induced-structure model, the authors argue . they argue that the nature of language has had a profound impact on progress in machine learning . |
On the Role of Pre-trained Language Models in Word Ordering: A Case Study with BART (2022.coling-1)
Copied to clipboard
| Challenge: | Existing work uses linear models and neural networks for word ordering, yet pre-trained language models have not been studied in word ordering. |
| Approach: | They propose a constrained language generation task using unordered words as input. |
| Outcome: | The proposed model is able to perform better than existing models and proves to be reliable. |
Aspect-Level Sentiment Analysis Via Convolution over Dependency Tree (D19-1)
Copied to clipboard
| Challenge: | Existing methods to identify sentiment polarity of opinion words are cumbersome due to the amount of opinionated material on the internet. |
| Approach: | They propose a method to identify sentiment polarity of opinion words on a specific aspect of a sentence using neural networks. |
| Outcome: | The proposed method is the state-of-the-art in aspect-based sentiment classification. |
Local Interpretation of Transformer Based on Linear Decomposition (2023.acl-long)
Copied to clipboard
| Challenge: | Existing work on local explanation generation attempts to understand model dynamics on word-level or phraselevel by assigning importance scores on input features. |
| Approach: | They propose to interpret neural networks by linear decomposition by a Transformer model on a single input and a linear decomposing of the output to generate local explanations. |
| Outcome: | The proposed method achieves competitive performance in sentiment classification and machine translation, and fidelity of explanation. |
Describe Me an Auklet: Generating Grounded Perceptual Category Descriptions (2023.emnlp-main)
Copied to clipboard
| Challenge: | Learning and using abstract perceptual concepts is under-investigated in the language-and-vision field. |
| Approach: | They propose a framework for testing category-level perceptual grounding in multi-modal language models by using separate neural networks to generate and interpret descriptions of visual categories. |
| Outcome: | The proposed framework compares prototype- and interpretation-based representations with the performance of the generation model and the interpretation model, which is an indicator of perceptual grounding. |
Evaluating and Enhancing the Robustness of Neural Network-based Dependency Parsing Models with Adversarial Examples (2020.acl-main)
Copied to clipboard
| Challenge: | Previously studies focused on semantic tasks such as sentiment analysis, question answering and reading comprehension. |
| Approach: | They propose two approaches to study where and how adversarial examples exist in dependency parsing . they use a state-of-the-art parser to find adversarials in existing texts . |
| Outcome: | The proposed approaches show that adversarial examples exist in dependency parsing . they show that up to 77% of input examples admit adversarials . |
NLProlog: Reasoning with Weak Unification for Question Answering in Natural Language (P19-1)
Copied to clipboard
| Challenge: | ambiguity in natural language is difficult to interpret due to large linguistic variability. |
| Approach: | They propose to use a Prolog prover to extend neural networks with logic programming to solve multi-hop reasoning tasks over natural language. |
| Outcome: | The proposed model outperforms baseline models on two question answering tasks and is competitive on the MedHop corpus. |
Interpretable Multimodal Misinformation Detection with Logic Reasoning (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for misinformation detection lack interpretability due to the black-box nature of the neural network. |
| Approach: | They propose a logic-based neural model which integrates interpretable logic clauses to express the reasoning process of the target task. |
| Outcome: | The proposed model can be generalizable across multiple misinformation sources and is based on three public datasets. |
Modeling Intra- and Inter-Modal Relations: Hierarchical Graph Contrastive Learning for Multimodal Sentiment Analysis (2022.coling-1)
Copied to clipboard
| Challenge: | Existing studies in Multimodal Sentiment Analysis lack a mechanism to understand complex relations between different modalities. |
| Approach: | They propose a hierarchical graph contrastive learning framework for multimodal sentiment analysis that explores the relationships between modality representations. |
| Outcome: | The proposed framework outperforms the state-of-the-art in multimodal sentiment analysis on two benchmark datasets. |
Language to Network: Conditional Parameter Adaptation with Natural Language Descriptions (2020.acl-main)
Copied to clipboard
| Challenge: | Experimental results show that N3 can out-perform previous natural-language based zero-shot learning methods across 4 different zero- shot image classification benchmarks. |
| Approach: | They propose a new paradigm for synthesizing task-specific neural networks from language descriptions and a generic pre-trained model from natural language. |
| Outcome: | The proposed model outperforms natural-language based zero-shot learning methods across 4 zero- shot image classification benchmarks. |
Accelerating Sparse Matrix Operations in Neural Networks on Graphics Processing Units (P19-1)
Copied to clipboard
| Challenge: | Graphics Processing Units (GPUs) are used to train and evaluate neural networks efficiently. |
| Approach: | They propose two new GPU algorithms for multiplying a matrix by a few-hot vector and fused softmax and top-N selection. |
| Outcome: | The proposed algorithms achieve speedups over state-of-the-art parallel GPU baselines of up to 7x and 50x, respectively. |
Transition-based Semantic Dependency Parsing with Pointer Networks (2020.acl-main)
Copied to clipboard
| Challenge: | Existing dependency parsers cannot be directly applied, so they need to be adaptable to deal with the absence of singlehead and connectedness constraints. |
| Approach: | They propose a transition system that produces labelled directed acyclic graphs and performs semantic dependency parsing with Pointer Networks. |
| Outcome: | The proposed system outperforms graph-based models and outperformed existing models on a harder NLP problem. |
The Treebank of Vedic Sanskrit (2020.lrec-1)
Copied to clipboard
| Challenge: | Vedic Sanskrit is a morphologically rich ancient Indian language of central importance for linguistic and historical research. |
| Approach: | They introduce the first treebank of Vedic Sanskrit, a morphologically rich ancient Indian language . they describe how sentences are annotated in the Universal Dependencies scheme and which syntactic constructions required special attention. |
| Outcome: | The proposed treebank reflects the development of metrical and prose texts over a period of 600 years. |
Learning to Copy for Automatic Post-Editing (D19-1)
Copied to clipboard
| Challenge: | Automatic post-editing (APE) is an important task in natural language processing. |
| Approach: | They propose a method that explicitly models how to copy words from a machine translation to a correct translation. |
| Outcome: | The proposed method outperforms all published methods on the WMT 2016-2017 datasets. |
Learning to Discover, Ground and Use Words with Segmental Neural Language Models (P19-1)
Copied to clipboard
| Challenge: | Existing models of word learning do not account for the long-range dependencies manifest in language and that are easily captured by recurrent neural networks. |
| Approach: | They propose a segmental neural language model that unifies word discovery, learning how words fit together to form sentences, and by conditioning the model on visual context, how words’ meanings ground in representations of nonlinguistic modalities. |
| Outcome: | The proposed model learns predictive distributions better than character LSTM models, discovers words competitively with nonparametric Bayesian word segmentation models, and improves on both. |
Don’t Search for a Search Method — Simple Heuristics Suffice for Adversarial Text Attacks (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent research has focused on adversarial text attacks on neural networks for natural language processing. |
| Approach: | They implement an algorithm inspired by zeroth order optimization-based attacks and compare it with benchmark results in TextAttack. |
| Outcome: | The proposed algorithm outperforms other black-box adversarial text attacks. |
Sequence Length is a Domain: Length-based Overfitting in Transformer Models (2021.emnlp-main)
Copied to clipboard
| Challenge: | Current Transformer-based sequence-to-sequence architectures can suffer from overfitting during training. |
| Approach: | They propose to use Transformer-based sequence-to-sequence architectures to overcome overfitting problems when generating very long sequences. |
| Outcome: | The proposed model performs worse on very long sequences than previous approaches on string editing and translation tasks when faced with sequences of length diverging from the length distribution in training data. |
Unraveling Feature Extraction Mechanisms in Neural Networks (2023.emnlp-main)
Copied to clipboard
| Challenge: | Neural networks have become indispensable across a variety of natural language processing tasks. |
| Approach: | They propose a theoretical approach based on Neural Tangent Kernels to investigate neural networks' internal mechanisms. |
| Outcome: | The proposed approach can be applied to analyze language modeling tasks . it shows that the choice of activation function can affect feature extraction . |
Avoiding the Hypothesis-Only Bias in Natural Language Inference via Ensemble Adversarial Training (2020.emnlp-main)
Copied to clipboard
| Challenge: | Neural models pick up on annotation artefacts and spurious correlations, resulting in learning sentences that suffer from the same biases. |
| Approach: | They propose to tackle this problem by using adversarial training to reduce the bias in sentence representations by using an ensemble of adversaries. |
| Outcome: | The proposed approach produces more robust models outperforming previous de-biasing efforts when generalised to 12 other NLI datasets. |
The Learnability of the Annotated Input in NMT Replicating (Vanmassenhove and Way, 2018) with OpenNMT (2020.lrec-1)
Copied to clipboard
| Challenge: | reproducibility of experiments is a key issue in Neural Networks, which are fed with variable samples of training data. |
| Approach: | They reproduce some of the experiments related to neural network training for Machine Translation as reported in . they annotated a sample from the EN-FR and EN-DE Europarl with syntactic and semantic annotations to train neural networks with the Nematus Neural Machine Translation toolkit. |
| Outcome: | The results obtained were lower than the original paper, but on a more limited set of annotations. |
Interventional Rationalization (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for rationalization use spurious correlations in data to compose rationales and make predictions. |
| Approach: | They propose a method to discover the causal rationales by using a structural causal model. |
| Outcome: | The proposed method is based on the causal theory and validates on three real-world datasets. |
Exploring Underexplored Limitations of Cross-Domain Text-to-SQL Generalization (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing text-to-SQL models do not generalize when faced with domain knowledge that does not frequently appear in training data. |
| Approach: | They propose a human-curated dataset based on the Spider benchmark for text-to-SQL translation. |
| Outcome: | The proposed model performs better on unseen domains than existing models on public benchmarks. |
Attention-Enhancing Backdoor Attacks Against BERT-based Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing textual backdoor attacks focus on generating stealthy triggers or modifying model weights. |
| Approach: | They propose a Trojan Attention Loss (TAL) which enhances the Trojan behavior by directly manipulating attention patterns. |
| Outcome: | The proposed method improves the effectiveness of the backdoor attacks on different backbone models and tasks. |
Bridging Perception, Memory, and Inference through Semantic Relations (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies suggest that it is impossible to learn meaning from surface form alone. |
| Approach: | They propose to develop triadic systems that combine neural and symbolic methods to provide a seamless information flow between them. |
| Outcome: | The proposed systems combine the strengths of neural and symbolic methods to achieve a seamless information flow between them. |
Named Entity Recognition Only from Word Embeddings (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing named entity recognition systems require large amounts of human annotated training data. |
| Approach: | They propose a fully unsupervised named entity recognition model which takes clues from pre-trained word embeddings. |
| Outcome: | The proposed model can be trained on two CoNLL benchmark datasets without annotating lexicon or corpus. |
Circuit Compositions: Exploring Modular Structures in Transformer-Based Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Recent advances in mechanistic interpretability have made progress in identifying circuits, the minimal computational subgraphs responsible for a model’s behavior on specific tasks. |
| Approach: | They propose to analyze circuits for highly compositional subtasks within a transformer-based language model to determine their modularity and how they relate to each other. |
| Outcome: | The proposed approach shows that the circuits identified exhibit notable node overlap and cross-task faithfulness. |
Bayesian Prompt Ensembles: Model Uncertainty Estimation for Black-Box Large Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to quantify uncertainty of pre-trained large language models require specific architectures or retraining strategies. |
| Approach: | They propose a Bayesian Prompts Ensemble approach to accurately quantify LLM uncertainty . they compute output probabilities through a weighted ensemble of different task instruction prompts . |
| Outcome: | The proposed approach achieves significantly superior calibration over baselines over a range of natural language classification tasks. |
Predicting the Focus of Negation: Model and Error Analysis (2020.acl-main)
Copied to clipboard
| Challenge: | Experimental results show that a scope detector can predict the focus of negation . negation is a complex phenomenon present in all human languages . |
| Approach: | They propose to leverage a scope detector to introduce the scope of negation as an additional input to the neural network. |
| Outcome: | The proposed model obtains the best results to date, and analyzes errors depending on scope and context information. |
Improving Quotation Attribution with Fictional Character Embeddings (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Recent methods to attribute quotes to human logic lack character representations, which often leads to errors in more challenging examples of attribution: anaphoric and implicit quotes. |
| Approach: | They propose to augment a popular quotation attribution system, BookNLP, with character embeddings that encode global stylistic information of characters derived from an off-the-shelf stylometric model, Universal Authorship Representation (UAR). |
| Outcome: | The proposed system improves anaphoric and implicit quotes, reaching state-of-the-art. |
Structured Tuning for Semantic Role Labeling (2020.acl-main)
Copied to clipboard
| Challenge: | Recent neural network-driven semantic role labeling systems have shown impressive improvements in F1 scores. |
| Approach: | They propose a framework to tune models using softened constraints only at training time. |
| Outcome: | The proposed framework outperforms the baseline model with minimal training time and consistent improvements under low-resource scenarios. |
Neuralizing Regular Expressions for Slot Filling (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to integrate neural networks and symbolic rules have their merits and weaknesses. |
| Approach: | They propose to integrate regular expressions into neural networks for a slot filling task . they use finite-state transducers to convert regular expression into a neural network . their model has superior zero-shot and few-shot performance . |
| Outcome: | The proposed model outperforms rules in zero-shot and few-shot scenarios and is competitive when training data is available. |
A Real-Time System for Credibility on Twitter (2020.lrec-1)
Copied to clipboard
| Challenge: | Using neural networks, we can analyze Twitter in real-time to determine whether users are credible and false. |
| Approach: | They propose to analyze Twitter in real-time using neural networks to determine credibility of tweets and users who posted them. |
| Outcome: | The proposed method analyzes Twitter in real-time to determine which users are credible and which are not, what is false or what is true on the Internet. |
Neural Networks Against (and For) Self-Training: Classification with Small Labeled and Large Unlabeled Sets (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing models for text classification suffer from the semantic drift problem, which is a problem for self-training. |
| Approach: | They propose a semi-supervised text classifier based on self-training using one positive and one negative property of neural networks. |
| Outcome: | The proposed model outperforms ten baseline models in five benchmarks and is additive to language model pretraining. |
Leveraging Hashtag Networks for Multimodal Popularity Prediction of Instagram Posts (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing popularity prediction approaches reduce hashtags to simple features such as hashtag length or number of hashtags in a post. |
| Approach: | They propose a multimodal framework to predict popular influencer posts on Instagram using post captions, image, hashtag network and topic model. |
| Outcome: | The proposed framework outperforms baseline models and unimodal models on popular influencer posts in Taiwan . it uses post captions, image, hashtag network, and topic model to predict popular influence post . |
Minding Language Models’ (Lack of) Theory of Mind: A Plug-and-Play Multi-Character Belief Tracker (2023.acl-long)
Copied to clipboard
| Challenge: | Empirical results show plug-and-play approach to reason about belief states of multiple characters in reading comprehension tasks is more precise and interpretable than previous approaches. |
| Approach: | They propose a plug-and-play approach to reason about the belief states of multiple characters in reading comprehension tasks via explicit symbolic representation. |
| Outcome: | The proposed algorithm improves theory of mind of off-the-shelf neural language models without supervision. |
Complex Numerical Reasoning with Numerical Semantic Pre-training Framework (2025.emnlp-main)
Copied to clipboard
| Challenge: | Numerical knowledge graphs (NKGs) are not limited to discrete entity-relation knowledge. |
| Approach: | They propose to combine numerical values and entities to solve multi-hop complex reasoning over incomplete knowledge graphs. |
| Outcome: | The proposed approach handles up to 102 types of complex numerical reasoning queries on three public datasets. |
From Isolation to Entanglement: When Do Interpretability Methods Identify and Disentangle Known Concepts? (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods to evaluate features disentangle concepts from activations of neural networks are limited by their quality . current methods for concept identification and steering are sparse autoencoders, but they are not reliable. |
| Approach: | They propose to evaluate how well featurization methods disentangle one concept from another . they use sentiment, domain, voice, and tense to steer these features . |
| Outcome: | The proposed evaluations show that featurization methods are insufficient to establish steering selectivity . the results suggest that steering a feature affects many concepts despite a near absence of interaction effects. |
Dynamic Head Selection for Neural Lexicalized Constituency Parsing (2025.acl-long)
Copied to clipboard
| Challenge: | Lexicalized parsing has traditionally been neglected in favor of unlexicalized, span-based methods. |
| Approach: | They propose a latent lexicalization framework that dynamically infers lexicals from data without relying on predefined head-finding rules. |
| Outcome: | The proposed model learns lexical dependencies directly from data, offering greater adaptability across languages and datasets. |
On the Correspondence between Compositionality and Imitation in Emergent Neural Communication (2023.findings-acl)
Copied to clipboard
| Challenge: | a study examining compositionality and imitation learning in a Lewis game demonstrates that it is difficult to imitate compositional languages. |
| Approach: | They explore the link between compositionality and imitation in a Lewis game . they show that the learning algorithm used to imitate is crucial . |
| Outcome: | The proposed model improves compositionality and imitation in a Lewis game . the study shows that compositional languages are easier to imitate . |
Distractor Generation in Multiple-Choice Tasks: A Survey of Methods, Datasets, and Evaluation (2024.emnlp-main)
Copied to clipboard
| Challenge: | Objective questions such as fill-in-the-blank and multiple-choice require examinees to select one valid answer from a set of invalid options. |
| Approach: | They examine distractor generation tasks, datasets, methods, and evaluation metrics for English objective questions. |
| Outcome: | The proposed task is based on fill-in-the-blank and multiple choice questions and is widely utilized in educational settings across various domains and subjects. |
Two Huge Title and Keyword Generation Corpora of Research Articles (2020.lrec-1)
Copied to clipboard
| Challenge: | Recent advances in sequence-to-sequence learning with neural networks have improved the quality of automatically generated text summaries and document keywords. |
| Approach: | They propose to use OAGSX and OAGKX datasets to analyze text summaries and document keywords. |
| Outcome: | The proposed models perform better than previous models on two large datasets . the authors hope to use the results to derive subsets of research articles from more disciplines . |
Summarization Beyond News: The Automatically Acquired Fandom Corpora (2020.lrec-1)
Copied to clipboard
| Challenge: | Abstractive summarization methods require large corpora to train neural architectures. |
| Approach: | They propose a novel automatic corpus construction approach that automatically constructs large open-licensed summarization corpora from existing large text collections and an evaluation process with human annotators. |
| Outcome: | The proposed approach can be used to train abstractive summarization models on large corpora and through a manual evaluation with human annotators. |
CoRRPUS: Code-based Structured Prompting for Neurosymbolic Story Understanding (2023.findings-acl)
Copied to clipboard
| Challenge: | Story generation and understanding has seen a surge in neurosymbolic work . symbolic methods are expensive and require a lot of time and expertise . |
| Approach: | They use Code-LLMs to bootstrap the use of symbolic methods for story understanding . they show that they can beat current LLM techniques on pre-existing stories with minimal hand engineering . |
| Outcome: | The proposed system beats state-of-the-art structured LLM techniques on pre-existing story understanding tasks with minimal hand engineering. |
SRF: Enhancing Document-Level Relation Extraction with a Novel Secondary Reasoning Framework (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for document-level relation extraction ignore bidirectional mention interaction when generating relational features for entity pairs. |
| Approach: | They propose a document-level relation extraction model that incorporates bidirectional mention fusion and a simple yet effective evidence extraction module for relation prediction. |
| Outcome: | The proposed model achieves SOTA performance and the proposed method is effective and general when integrated into existing models. |
Model Interpretability and Rationale Extraction by Input Mask Optimization (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for creating explanations for black-box models struggle with deriving easily interpretable explanations. |
| Approach: | They propose a model-agnostic method to generate extractive explanations for neural network predictions using masking parts of the input that the model does not consider indicative of the respective class. |
| Outcome: | The proposed method achieves state-of-the-art results in a paragraph-level rationale extraction task, showing that this task can be performed without training a specialized model. |
LCGbank: A Corpus of Syntactic Analyses Based on Proof Nets (2024.lrec-main)
Copied to clipboard
| Challenge: | Recent studies have focused on statistical syntactic parsing with proof nets . however, there has been a paucity of corpora in formalisms for which proof net is applicable . |
| Approach: | They propose a corpus of syntactic analyses based on Lambek categorial grammar . they leverage the relationship between LCG and CCG to address this problem . |
| Outcome: | The proposed method exploits the relationship between LCG and CCG to build an English-language corpus of syntactic analyses based on proof nets . the results suggest that the proposed method is weakly context-free equivalent and NP-complete . |
SLIDE: A Framework Integrating Small and Large Language Models for Open-Domain Dialogues Evaluation (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to evaluate open domain dialogues have a one-to-many problem . existing approaches lack commonsense reasoning biases and perform poorly in domain-specific scenarios. |
| Approach: | They propose a framework that leverages both a small, specialised model and LLMs for the evaluation of open-domain dialogues. |
| Outcome: | The proposed framework achieves state-of-the-art performance in both classification and evaluation tasks and exhibits better correlation with human judgements. |
Assessing the influence of attractor-verb distance on grammatical agreement in humans and language models (2023.emnlp-main)
Copied to clipboard
| Challenge: | a subject-verb agreement in the presence of an attractor noun is a complex behavior . formal linguistic theories postulate the existence of an underlying structure that governs language processing . |
| Approach: | They hypothesize that the attractor-verb agreement may be a factor in grammatical decision-making . they hypothesized that classical models of attraction might suffice to explain this phenomenon . |
| Outcome: | The proposed model improves on humans and artificial neural networks while keeping the length of the sentence equal. |
ShadowLLM: Predictor-based Contextual Sparsity for Large Language Models (2024.emnlp-main)
Copied to clipboard
Yash Akhauri, Ahmed AbouElhamayed, Jordan Dotzel, Zhiru Zhang, Alexander Rush, Safeen Huda, Mohamed Abdelfattah
| Challenge: | Prior work has focused on contextual sparsity, but it has not been successful. |
| Approach: | They propose a novel pruning predictor that can shadow the LLM behavior and enforce better sparsity patterns. |
| Outcome: | The proposed model can shadow the LLM behavior and enforce better sparsity patterns, resulting in 15% improvement in end-to-end accuracy compared to prior methods. |
Initialization of Large Language Models via Reparameterization to Mitigate Loss Spikes (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to train large language models that require a non-uniform model norm are not effective. |
| Approach: | They propose a technique that allows for uniformity of the norm of the model parameters . they propose 'weight scaling as reparameterization' to adjust the norm to the parameter . |
| Outcome: | The proposed technique outperforms existing methods and stabilizes training with the transformer decoders. |
Sentence Smith: Controllable Edits for Evaluating Text Embeddings (2025.emnlp-main)
Copied to clipboard
| Challenge: | Controllable and transparent text generation has been a long-standing goal in NLP . but previous approaches were hindered by parsing and generation insufficiencies . |
| Approach: | They propose a framework for English that has three steps: 1. Parsing a sentence into a semantic graph. 2. Applying human-designed semantic manipulation rules. 3. Generating text from the manipulated graph. |
| Outcome: | The proposed framework for English is based on a neural network and parsers. |
The Ethical Question – Use of Indigenous Corpora for Large Language Models (2024.lrec-main)
Copied to clipboard
Linda Wiechetek, Flammie A. Pirinen, Børre Gaup, Trond Trosterud, Maja Lisa Kappfjell, Sjur Moshagen
| Challenge: | Creating language technology based on language data is becoming more popular . indigenous language resources are not comparable in that they would encode the most recent normativised language . |
| Approach: | They describe an ethical way to work with indigenous languages based on language data . they say data driven methods make assumptions based upon majority languages they work with . authors say data-driven methods are not ethical or beneficial . |
| Outcome: | The proposed method is ethical and sustainable, and can be applied to indigenous languages in an ethical way. |
AgentSlimming: Towards Efficient and Cost-Aware Multi-Agent Systems (2026.acl-long)
Copied to clipboard
| Challenge: | Automated expansion methods often result in bloated structures with redundant agents, leading to excessive token consumption. |
| Approach: | They propose a plug-and-play compression framework for graph-structured multi-agent workflows . they estimate the importance score of each agent and remove redundant agents . |
| Outcome: | Experiments show that AgentSlimming reduces average token cost by 78.9% with negligible performance degradation. |
Towards Robust Temporal Activity Localization Learning with Noisy Labels (2024.lrec-main)
Copied to clipboard
Daizong Liu, Xiaoye Qu, Xiang Fang, Jianfeng Dong, Pan Zhou, Guoshun Nan, Keke Tang, Wanlong Fang, Yu Cheng
| Challenge: | Existing methods for temporal activity localization are expensive and difficult to satisfy due to subjective labeling. |
| Approach: | They propose a new TAL setting where a TAL model should be robust to mixed training data with noisy moment boundaries. |
| Outcome: | The proposed method is significantly more robust to noisy training data than existing methods. |
Massively Multilingual Joint Segmentation and Glossing (2026.acl-long)
Copied to clipboard
Michael Ginn, Lindia Tjuatja, Enora Rice, Ali Marashian, Maria Valentini, Jasmine Xu, Graham Neubig, Alexis Palmer
| Challenge: | Existing models generate morpheme-level glosses but assign them to whole words without predicting the actual morphological boundaries, making them less interpretable and therefore untrustworthy to human annotators. |
| Approach: | They propose to use neural networks to predict interlinear glosses and morphological segmentation from raw text. |
| Outcome: | The proposed model outperforms GlossLM on glossing and beats open-source models on segmentation, glossing, and alignment. |
Understanding the Prompt Sensitivity (2026.acl-long)
Copied to clipboard
| Challenge: | Prompt sensitivity is a measure of how strongly the output of a large language model (LLM) depends on the exact wording of its input prompt. |
| Approach: | They consider LLMs as multivariate functions and perform a first-order Taylor expansion to analyze the relationship between meaning-preserving prompts, their gradients, and log probabilities of the model’s next token. |
| Outcome: | The proposed model disperses meaning-preserving inputs, making it difficult to reduce to 0. The proposed models also dispersing prompt variants are more likely to introduce prompt sensitivity risks in LLMs. |